Fabrice Matulic
ABSTRACT
Querying geographical data on map applications running on touch devices is mainly performed by typing queries using virtual keyboards. Some of those devices are additionally equipped with styli to facilitate freehand sketching and an-notating. As shown by prior work, such hand-drawn sketches can also be used for intuitive and effective spatial querying of geographical data. Building on that groundwork, we present a set of pen-based techniques to selectively convert map annotations into spatial queries with implicitly or explicitly specified scopes. We show how those techniques can be used for trip-planning tasks involving route-finding and searching of points of interest. In a controlled user study comparing the usability and efficiency of the techniques for different querying patterns, we establish participants' general preference for explicit input scopes and obtain indications that, provided handwriting is correctly recognised, input times are comparable to that of a standard (soft) keyboard-based interface. Based on those results and participant feedback, we propose a number of enhancements and extensions to inform the design of future pen-based map applications.
Supplemental Material
- GlobalWebIndex Device Report Q2 2014.Google Scholar
- Blaser, A. D. and Egenhofer, M. J. A visual tool for querying geographic databases. In Proc. AVI '00, 211--216. Google ScholarDigital Library
- Brandl, P., Forlines, C., Wigdor, D., Haller, M. and Shen, C. Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In Proc. AVI 2008, 154--161. Google ScholarDigital Library
- Cheyer, A. and Julia, L. Multimodal maps: An agent-based approach. In Multimodal Human-Computer Communication, 1374, (1998), 111--121. Google ScholarDigital Library
- Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L. and Clow, J. QuickSet: multimodal interaction for distributed applications. In Proc. MULTIMEDIA '97, 31--40. Google ScholarDigital Library
- Doyle, J., Bertolotto, M. and Wilson, D. A Survey of Multimodal Interfaces for Mobile Mapping Applications. In Mapbased Mobile Services, (2008), 146--167.Google ScholarCross Ref
- Ehlen, P. and Johnston, M. Multimodal interaction patterns in mobile local search. In Proc. IUI '12, 21--24. Google ScholarDigital Library
- Ferri, F., Grifoni, P. and Rafanelli, M. Querying by Sketch Geographical Databases and Ambiguities. In Database and Expert Systems Applications, 3588, (2005), 524--533. Google ScholarDigital Library
- Forbus, K. D., Usher, J. and Chapman, V. Qualitative spatial reasoning about sketch maps. AI magazine, 25, 3 (2004), 6172. Google ScholarDigital Library
- Hamilton, W., Kerne, A. and Robbins, T. High-performance pen + touch modality interactions: a real-time strategy game eSports context. In Proc. UIST 2012, 309--318. Google ScholarDigital Library
- Hinckley, K., Baudisch, P., Ramos, G. and Guimbretiere, F. Design and analysis of delimiters for selection-action pen gesture phrases in scriboli. In Proc. CHI '05, 451--460. Google ScholarDigital Library
- Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H. and Buxton, B. Pen + touch = new tools. In Proc. UIST 2010, 27--36. Google ScholarDigital Library
- Hinckley, K., Zhao, S., Sarin, R., Baudisch, P., Cutrell, E., Shilman, M. and Tan, D. InkSeine: In Situ search for active note taking. In Proc. CHI 2007, 251--260. Google ScholarDigital Library
- Hui, P.-Y. and Meng, H. M. Joint interpretation of input speech and pen gestures for multimodal human-computer interaction. In Proc. INTERSPEECH 2006, 1197--1200.Google Scholar
- Hurtig, T. A mobile multimodal dialogue system for public transportation navigation evaluated. In Proc. MobileHCI '06, 251--254. Google ScholarDigital Library
- Johnston, M., Bangalore, S., Vasireddy, G., Stent, A., Ehlen, P., Walker, M., Whittaker, S. and Maloor, P. MATCH: an architecture for multimodal dialogue systems. In Proc. ACL '02, 376--383. Google ScholarDigital Library
- Jokinen, K. User Interaction in Mobile Navigation Applications. In Map-based Mobile Services, (2008), 168--197.Google ScholarCross Ref
- Krüger, A., Butz, A., Müller, C., Stahl, C., Wasinger, R., Steinberg, K.-E. and Dirschl, A. The connected user interface: realizing a personal situated navigation service. In Proc. IUI '04, 161--168. Google ScholarDigital Library
- Matulic, F. and Norrie, M. Pen and Touch Gestural Environment for Document Editing on Interactive Tabletops. In Proc. ITS 2013, 41--50. Google ScholarDigital Library
- Oviatt, S., DeAngeli, A. and Kuhn, K. Integration and synchronization of input modes during multimodal humancomputer interaction. In Proc. CHI '97, 415--422. Google ScholarDigital Library
- Silva, G. C. d. and Aizawa, K. Retrieving multimedia travel stories using location data and spatial queries. In Proc. MM '09, 785--788. Google ScholarDigital Library
- Vogel, D., Cudmore, M., Casiez, G., Balakrishnan, R. and Keliher, L. Hand occlusion with tablet-sized direct pen input. In Proc. CHI '09, 557--566. Google ScholarDigital Library
- Willems, D. J. and Vuurpijl, L. G. Pen gestures in online map and photograph annotation tasks. In Proc. IWFHR 2006, 297402.Google Scholar
Index Terms
- Spatial Querying of Geographical Data with Pen-Input Scopes
Recommendations
Towards document engineering on pen and touch-operated interactive tabletops
UIST Adjunct Proceedings '12: Adjunct proceedings of the 25th annual ACM symposium on User interface software and technologyTouch interfaces have now become mainstream thanks to modern smartphones and tablets. However, there are still very few "productivity" applications, i.e. tools that support mundane but essential work, especially for large interactive surfaces such as ...
Terrain Modelling with a Pen & Touch Tablet and Mid-Air Gestures in Virtual Reality
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsWorld building or terrain modelling is an essential task when designing games, natural simulations or artistic creations involving virtual 3D landscapes. To support this task, we propose a virtual reality (VR) system based on a pen and touch tablet used ...
Pen and touch gestural environment for document editing on interactive tabletops
ITS '13: Proceedings of the 2013 ACM international conference on Interactive tabletops and surfacesCombined pen and touch input is an interaction paradigm attracting increasing interest both in the research community and recently in industry. In this paper, we illustrate how pen and touch interaction techniques can be leveraged for editing and ...
Comments