Khalad Hasan
David Ahlström
ABSTRACT
In recent years, around device input has emerged as a complement to standard touch input, albeit in limited tasks and contexts, such as for item selection or map navigation. We push the boundaries for around device interactions to facilitate an entire smartphone application: browsing through large information lists to make a decision. To this end, we present AirPanes, a novel technique that allows two-handed in-air interactions, conjointly with touch input to perform analytic tasks, such as making a purchase decision. AirPanes resolves the inefficiencies of having to switch between multiple views or panes in common smartphone applications. We explore the design factors that make AirPanes efficient. In a controlled study, we find that AirPanes is on average 50% more efficient that standard touch input for an analytic task. We offer recommendations for implementing AirPanes in a broad range of applications.
Supplemental Material
Available for Download
zip
pn4727-file4.zip (584 B)
The file contains closed captions for the video preview.
- David Ahlström, Khalad Hasan, and Pourang Irani. 2014. Are you comfortable doing that?: acceptance studies of around-device gestures in and for public settings. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '14), 193--202. https://doi.org/10.1145/2628363.2628381Google Scholar
Digital Library
- Alexa Top 500 Global Sites. 2017. Retrieved January 04, 2017 from http://www.alexa.com/topsites.Google Scholar
- Ravin Balakrishnan and Ken Hinckley. 2000. Symmetric bimanual interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '00), 33--40. https://doi.org/10.1145/332040.332404Google ScholarDigital Library
- Alex Butler, Shahram Izadi, and Steve Hodges. 2008. SideSight: multi-'touch' interaction around small devices. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08), 201--204. http://dl.acm.org/citation.cfm?doid=1449715.1449746Google ScholarDigital Library
- W. Buxton and B. Myers. 1986. A study in two-handed input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '86), 321--326. https://doi.org/10.1145/22627.22390Google ScholarDigital Library
- Juan Pablo Carrascal and Karen Church. 2015. An in-situ study of mobile app & mobile search interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2739--2748. https://doi.org/10.1145/2702123.2702486Google ScholarDigital Library
- Didier Casalta, Yves Guiard, and Michel Beaudouin-Lafon. 1999. Evaluating two-handed input techniques: Rectangle editing and navigation. In Proceedings of the CHI '99 Extended Abstracts on Human Factors in Computing Systems (CHI EA '99), 236--237. https://doi.org/10.1145/632716.632862Google ScholarDigital Library
- Youli Chang, Sehi L'Yi, Kyle Koh, and Jinwook Seo. 2015. Understanding users' touch behavior on large mobile touch-screens and assisted targeting by tilting gesture. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 1499--1508. https://doi.org/10.1145/2702123.2702425Google ScholarDigital Library
- Xiang 'Anthony' Chen, Julia Schwarz, Chris Harrison, Jennifer Mankoff, and Scott E. Hudson. 2014. Air+touch: interweaving touch & in-air gestures. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14), 519--525. https://doi.org/10.1145/2642918.2647392Google ScholarDigital Library
- Euan Freeman, Stephen Brewster, and Vuokko Lantz. 2016. Do that, there: an interaction technique for addressing in-air gesture systems. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 2319--2331. http://dx.doi.org/10.1145/2858036.2858308 Google ScholarDigital Library
- Verena Giller, Rudolf Melcher, Johann Schrammel, Reinhard Sefelin, and Manfred Tscheligi. 2003. Usability evaluations for multi-device application development three example studies. In Proceedings of Human-Computer Interaction with Mobile Devices and Services (MobileHCI '03), 302--316. http://link.springer.com/chapter/10.1007/978-3-540-45233-1_22Google ScholarCross Ref
- Google Play. 2017. Retrieved January 04, 2017 from https://play.google.com/store?hl=enGoogle Scholar
- Yves Guiard. 1987. Asymmetric division of labor in human skilled bimanual action. Journal of Motor Behavior 19, 4: 486--517. https://doi.org/10.1080/00222895.1987.10735426 Google ScholarCross Ref
- Sean Gustafson, Patrick Baudisch, Carl Gutwin, and Pourang Irani. 2008. Wedge: clutter-free visualization of off-screen locations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08), 787--796. https://doi.org/10.1145/1357054.1357179Google ScholarDigital Library
- Sean Gustafson, Daniel Bierwirth, and Patrick Baudisch. 2010. Imaginary interfaces: spatial interaction with empty hands and without visual feedback. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (UIST '10), 3--12. http://dl.acm.org/citation.cfm?doid=1866029.1866033Google ScholarDigital Library
- Sean G. Gustafson and Pourang P. Irani. 2007. Comparing visualizations for tracking off-screen moving targets. In Proceedings of the CHI '07 Extended Abstracts on Human Factors in Computing Systems (CHI EA '07), 2399--2404. https://doi.org/10.1145/1240866.1241014Google ScholarDigital Library
- Chris Harrison and Scott E. Hudson. 2009. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09), 121--124. https://doi.org/10.1145/1622176.1622199Google ScholarDigital Library
- Khalad Hasan, David Ahlström, and Pourang Irani. 2013. Ad-binning: leveraging around device space for storing, browsing and retrieving mobile device content. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 899--908. http://dl.acm.org/citation.cfm?doid=2470654.2466115Google ScholarDigital Library
- Khalad Hasan, David Ahlström, and Pourang Irani. 2015. SAMMI: a spatially-aware multi-mobile interface for analytic map navigation tasks. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15), 36--45. https://doi.org/10.1145/2785830.2785850Google ScholarDigital Library
- Khalad Hasan, Junhyeok Kim, David Ahlström, and Pourang Irani. 2016. Thumbs-Up: 3d spatial thumb-reachable space for one-handed thumb interaction on smartphones. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16), 103--106. https://doi.org/10.1145/2983310.2985755Google ScholarDigital Library
- Khalad Hasan, Xing-Dong Yang, Hai-Ning Liang, and Pourang Irani. 2012. How to position the cursor?: an exploration of absolute and relative cursor positioning for back-of-device input. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '12), 103--112. https://doi.org/10.1145/2371574.2371591Google ScholarDigital Library
- Doris Hausen, Sebastian Boring, and Saul Greenberg. 2013. The unadorned desk: exploiting the physical space around a display as an input canvas. In Proceedings of the 14th IFIP TC13 Conference on Human-Computer Interaction (INTERACT '13), 140--158. http://link.springer.com/chapter/10.1007/978-3-642-40483-2_10Google ScholarCross Ref
- Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1063--1072. https://doi.org/10.1145/2556288.2557130Google ScholarDigital Library
- David Holman, Andreas Hollatz, Amartya Banerjee, and Roel Vertegaal. 2013. Unifone: designing for auxiliary finger input in one-handed mobile interactions. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI '13), 177--184. http://dx.doi.org/10.1145/2460625.2460653 Google ScholarDigital Library
- Brett Jones, Rajinder Sodhi, David Forsyth, Brian Bailey, and Giuliano Maciocci. 2012. Around device interaction for multiscale navigation. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '12), 83--92. https://doi.org/10.1145/2371574.2371589Google ScholarDigital Library
- Amy K. Karlson and Benjamin B. Bederson. 2007. ThumbSpace: generalized one-handed input for touchscreen-based mobile devices. In Proceedings of the 11th IFIP TC 13 International Conference on Human-Computer Interaction (INTERACT'07), 324--338. http://dx.doi.org/10.1007/978-3-540-74796-3_30 Google ScholarCross Ref
- Amy K. Karlson, Benjamin B. Bederson, and Jose L. Contreras-Vidal. 2008. Understanding one handed use of mobile devices. In Handbook of Research on User Interface Design and Evaluation for Mobile Technology, 86--101. Google ScholarCross Ref
- Amy K. Karlson, Benjamin B. Bederson, and John SanGiovanni. 2005. AppLens and launchTile: two designs for one-handed thumb use on small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05), 201--210. https://doi.org/10.1145/1054972.1055001Google ScholarDigital Library
- Hamed Ketabdar, Mehran Roshandel, and Kamer Ali Yüksel. 2010. MagiWrite: towards touchless digit entry using 3D space around mobile devices. In Proceedings of the 12th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '10), 443--446. https://doi.org/10.1145/1851600.1851701Google ScholarDigital Library
- Sven Kratz and Michael Rohs. 2009. Hoverflow: exploring around-device interaction with IR distance sensors. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09), Article 42, 4 pages. https://doi.org/10.1145/1613858.1613912Google ScholarDigital Library
- Sven Kratz, Michael Rohs, Dennis Guse, Jörg Müller, Gilles Bailly, and Michael Nischt. 2012. PalmSpace: continuous around-device gestures vs. multitouch for 3D rotation tasks on mobile devices. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '12), 181--188. https://doi.org/10.1145/2254556.2254590Google ScholarDigital Library
- Jianwei Lai and Dongsong Zhang. 2014. ExtendedThumb: a motion-based virtual thumb for improving one-handed target acquisition on touch-screen mobile devices. In Proceedings of the Extended Abstracts of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI EA '14), 1825--1830. https://doi.org/10.1145/2559206.2581158Google ScholarDigital Library
- Leap motion. 2017. Retrieved January 04, 2017 from https://www.leapmotion.com/Google Scholar
- Andrea Leganchuk, Shumin Zhai, and William Buxton. 1998. Manual and cognitive benefits of two-handed input: an experimental study. ACM Transaction on Human-Computer Interaction. 5, 4: 326--359. https://doi.org/10.1145/300520.300522Google ScholarDigital Library
- Mobile eCommerce Stats in 2016 and the future. 2017. Retrieved January 04, 2017 from http://www.outerboxdesign.com/web-design-articles/mobile-ecommerce-statisticsGoogle Scholar
- Mobile Marketing Statistics Compilation. 2017. Retrieved January 04, 2017 from http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/Google Scholar
- Motion Capture System | VICON. 2017. Retrieved January 04, 2017 from https://www.vicon.com/.Google Scholar
- Takehiro Niikura, Yuki Hirobe, Alvaro Cassinelli, Yoshihiro Watanabe, Takashi Komuro, and Masatoshi Ishikawa. 2010. In-air typing interface for mobile devices with vibration feedback. In ACM SIGGRAPH 2010 Emerging Technologies (SIGGRAPH '10), Article 15, 1 pages. https://doi.org/10.1145/1836821.1836836Google ScholarDigital Library
- Michel Pahud, Ken Hinckley, Shamsi Iqbal, Abigail Sellen, and Bill Buxton. 2013. Toward compound navigation tasks on mobiles via spatial manipulation. In Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '13), 113--122. https://doi.org/10.1145/2493190.2493210Google ScholarDigital Library
- Project Soli. 2017. Retrieved January 04, 2017 from https://atap.google.com/soli/Google Scholar
- Jie Song, Gábor Sörös, Fabrizio Pece, Sean Ryan Fanello, Shahram Izadi, Cem Keskin, and Otmar Hilliges. 2014. In-air gestures around unmodified mobile devices. In Proceedings of the 27th Annual ACM symposium on User Interface Software and Technology (UIST '14), 319--329. https://doi.org/10.1145/2642918.2647373Google ScholarDigital Library
- Martin Spindler, Martin Schuessler, Marcel Martsch, and Raimund Dachselt. 2014. Pinch-drag-flick vs. spatial input: rethinking zoom & pan on mobile displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1113--1122. https://doi.org/10.1145/2556288.2557028Google ScholarDigital Library
- Structure sensor. 2017. Retrieved January 04, 2017 from http://structure.io/Google Scholar
- Eduardo Velloso, Jayson Turner, Jason Alexander, Andreas Bulling and Hans Gellersen. 2015. An empirical investigation of gaze selection in mid-air gestural 3d manipulation. In Proceedings of the 15th IFIP TC13 Conference on Human-Computer Interaction (INTERACT '13), 315--330. http://dx.doi.org/10.1007/978-3-319-22668-2_25 Google ScholarDigital Library
- Anusha Withana, Roshan Peiris, Nipuna Samarasekara, and Suranga Nanayakkara. 2015. zSense: enabling shallow depth gesture recognition for greater input expressivity on smart wearables. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3661--3670. http://dx.doi.org/10.1145/2702123.2702371 Google ScholarDigital Library
- Xing-Dong Yang, Khalad Hasan, Neil Bruce, and Pourang Irani. 2013. Surround-see: enabling peripheral vision on smartphones during active use. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13), 291--300. https://doi.org/10.1145/2501988.2502049Google ScholarDigital Library
- Chen Zhao, Ke-Yu Chen, Md Tanvir Islam Aumi, Shwetak Patel, and Matthew S. Reynolds. 2014. SideSwipe: detecting in-air gestures around mobile devices using actual GSM signal. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14), 527--534. https://doi.org/10.1145/2642918.2647380Google ScholarDigital Library
Index Terms
- AirPanes: Two-Handed Around-Device Interaction for Pane Switching on Smartphones
Recommendations
Exploring Boundless Scroll by Extending Motor Space
MobileHCI '15: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and ServicesThis paper explores the Boundless Scroll interface that extends the motor space for finger drag motions into the off-screen space using a tracking system around the screen. Seamlessly extended motor spaces enable continuous scroll actions across the on- ...
Thumbs-Up: 3D Spatial Thumb-Reachable Space for One-Handed Thumb Interaction on Smartphones
SUI '16: Proceedings of the 2016 Symposium on Spatial User InteractionPeople very often use mobile devices with one hand to free the second hand for other tasks. In such cases, the thumb of the hand holding the device is the only available input finger, making multi-touch interactions impossible. Complicating interaction ...
Ether-Toolbars: Evaluating Off-Screen Toolbars for Mobile Interaction
IUI '18 Companion: Companion Proceedings of the 23rd International Conference on Intelligent User InterfacesIn mobile interaction, the use of touchscreen interaction, while beneficial from the perspective of portability, has limited spatial accuracy due to the "fat finger problem". As a result, an important challenge on mobile interaction is to find solutions ...
Comments