research-article

Porous Interfaces for Small Screen Multitasking using Finger Identification

Authors:

Muhammed Anwar,

Ravin BalakrishnanAuthors Info & Claims

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

Pages 145 - 156

https://doi.org/10.1145/2984511.2984557

Published: 16 October 2016 Publication History

Abstract

The lack of dedicated multitasking interface features in smartphones has resulted in users attempting a sequential form of multitasking via frequent app switching. In addition to the obvious temporal cost, it requires physical and cognitive effort which increases multifold as the back and forth switching becomes more frequent. We propose porous interfaces, a paradigm that combines the concept of translucent windows with finger identification to support efficient multitasking on small screens. Porous interfaces enable partially transparent app windows overlaid on top of each other, each of them being accessible simultaneously using a different finger as input. We design porous interfaces to include a broad range of multitasking interactions with and between windows, while ensuring fidelity with the existing smartphone interactions. We develop an end-to-end smartphone interface that demonstrates porous interfaces. In a qualitative study, participants found porous interfaces intuitive, easy, and useful for frequent multitasking scenarios.

Supplementary Material

suppl.mov (uist3356-file3.mp4)

Supplemental video

Download
103.02 MB

MP4 File (p145-gupta.mp4)

Download
267.13 MB

References

[1]

Au, O.K.-C. and Tai, C.-L. 2010. Multitouch finger registration and its applications. Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction - OZCHI '10 (New York, New York, USA, Nov. 2010), 41.

Digital Library

[2]

Banovic, N., Brant, C., Mankoff, J. and Dey, A. 2014. ProactiveTasks. Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services MobileHCI '14 (New York, New York, USA, Sep. 2014), 243--252.

Digital Library

[3]

Benko, H., Saponas, T.S., Morris, D. and Tan, D. 2009. Enhancing input on and above the interactive surface with muscle sensing. Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '09 (New York, New York, USA, Nov. 2009), 93.

Digital Library

[4]

Bier, E.A., Stone, M.C., Pier, K., Buxton, W. and DeRose, T.D. 1993. Toolglass and magic lenses. Proceedings of the 20th annual conference on Computer graphics and interactive techniques SIGGRAPH '93 (New York, New York, USA, Sep. 1993), 73--80.

Digital Library

[5]

Böhmer, M., Hecht, B., Schöning, J., Krüger, A. and Bauer, G. 2011. Falling asleep with Angry Birds, Facebook and Kindle. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services MobileHCI '11 (New York, New York, USA, Aug. 2011), 47.

Digital Library

[6]

Choi, K., Song, H., Koh, K., Bok, J. and Jinwook, S. 2016. Peek-a-View: Smartphone Cover Interaction for Multi-tasking. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '16 (2016).

Digital Library

[7]

Ewerling, P., Kulik, A. and Froehlich, B. 2012. Finger and hand detection for multi-touch interfaces based on maximally stable extremal regions. Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces ITS '12 (New York, New York, USA, Nov. 2012), 173.

Digital Library

[8]

Fitzpatrick, G.P., Haynes, T.R. and Williams, M.L. 1995. Method and apparatus for accessing touch screen desktop objects via fingerprint recognition. 1995.

[9]

Goguey, A., Casiez, G., Vogel, D., Chevalier, F., Pietrzak, T. and Roussel, N. 2014. A three-step interaction pattern for improving discoverability in finger identification techniques. Proceedings of the adjunct publication of the 27th annual ACM symposium on User interface software and technology - UIST'14 Adjunct (New York, New York, USA, Oct. 2014), 33--34.

Digital Library

[10]

Gupta, A. and Balakrishnan, R. 2016. DualKey: Miniature Screen Text Entry via Finger Identification. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '16 (San Jose, California, USA, 2016).

Digital Library

[11]

Harrison, B.L., Ishii, H., Vicente, K.J. and Buxton, W.A.S. 1995. Transparent layered user interfaces. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '95 (New York, New York, USA, May 1995), 317--324.

Digital Library

[12]

Harrison, B.L., Kurtenbach, G. and Vicente, K.J. 1995. An experimental evaluation of transparent user interface tools and information content. Proceedings of the 8th annual ACM symposium on User interface and software technology - UIST '95 (New York, New York, USA, Dec. 1995), 81--90.

Digital Library

[13]

Harrison, B.L. and Vicente, K.J. 1996. An experimental evaluation of transparent menu usage. Proceedings of the SIGCHI conference on Human factors in computing systems common ground - CHI '96 (New York, New York, USA, Apr. 1996), 391--398.

Digital Library

[14]

Holz, C. and Baudisch, P. 2013. Fiberio. Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13 (New York, New York, USA, Oct. 2013), 41--50.

Digital Library

[15]

Hoober, S. 2013. How do users really hold mobile devices. UXmatters, http://www. uxmatters. com/mt/archives/. (2013).

[16]

Ishak, E.W. and Feiner, S.K. 2004. Interacting with hidden content using content-aware free-space transparency. Proceedings of the 17th annual ACM symposium on User interface software and technology - UIST '04 (New York, New York, USA, Oct. 2004), 189.

Digital Library

[17]

Kamba, T., Elson, S.A., Harpold, T., Stamper, T. and Sukaviriya, P. 1996. Using small screen space more efficiently. Proceedings of the SIGCHI conference on Human factors in computing systems common ground - CHI '96 (New York, New York, USA, Apr. 1996), 383--390.

Digital Library

[18]

Kandogan, E. and Shneiderman, B. 1997. Elastic Windows. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '97 (New York, New York, USA, Mar. 1997), 250--257.

Digital Library

[19]

Leiva, L., Böhmer, M., Gehring, S. and Krüger, A. 2012. Back to the app. Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services MobileHCI '12 (New York, New York, USA, Sep. 2012), 291.

Digital Library

[20]

Marquardt, N., Kiemer, J., Ledo, D., Boring, S. and Greenberg, S. 2011. Designing user-, hand-, and handpart-aware tabletop interactions with the TouchID toolkit. Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '11 (New York, New York, USA, Nov. 2011), 21.

Digital Library

[21]

Multitasking on Mobile Devices: 2015. https://www.nngroup.com/articles/multitaskingmobile/. Accessed: 2016-03-21.

[22]

Nagata, S.F. 2003. Multitasking and Interruptions during Mobile Web Tasks. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 47, 11 (Oct. 2003), 1341--1345.

[23]

Oakley, I., Lee, D., Islam, M.R. and Esteves, A. 2015. Beats. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15 (New York, New York, USA, Apr. 2015), 1237--1246.

Digital Library

[24]

Roy, Q., Guiard, Y., Bailly, G., Lecolinet, É. and Rioul, O. 2015. Glass+Skin: An Empirical Evaluation of the Added Value of Finger Identification to Basic Single-Touch Interaction on Touch Screens. INTERACT: IFIP International Conference on Human-Computer Interaction (2015).

[25]

Salvucci, D., Kushleyeva, Y. and Lee, F. 2004. Toward an ACT-R General Executive for Human Multitasking. ICCM. (2004).

[26]

Sonavation has bonded 3D fingerprint sensors to Gorilla Glass: 2015. http://thenextweb.com/insider/2015/07/21/sonovatio n-has-bonded-3d-fingerprint-sensors-to-gorillaglass-kiss-your-home-button-goodbye/. Accessed: 2015-09-11.

[27]

Spink, A., Cole, C. and Waller, M. 2009. Multitasking behavior. Annual Review of Information Science and Technology. 42, 1 (Nov. 2009), 93--118.

Digital Library

[28]

Sugiura, A. and Koseki, Y. 1998. A user interface using fingerprint recognition. Proceedings of the 11th annual ACM symposium on User interface software and technology - UIST '98 (New York, New York, USA, Nov. 1998), 71--79.

Digital Library

[29]

Wagner, J., Lecolinet, E. and Selker, T. 2014. Multi-finger chords for hand-held tablets. Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14 (New York, New York, USA, Apr. 2014), 2883--2892.

Digital Library

[30]

Westerman, W.C. and Elias, J.G. 2006. System and method for packing multi-touch gestures onto a hand. 2006.

[31]

Yousefpor, M. and Bussat, J. 2014. Fingerprint Sensor in an Electronic Device. US Patent App. 14/ 451,076. (2014).

Cited By

Lu YDing DWang RXue GKostakos VKay JHoang T(2024)HCMG: Human-Capacitance based Micro Gesture for VR/ARCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678386(766-770)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678386
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Huang ZGao CWang HDeng XLai YMa CQin SLiu YWang H(2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643559
Show More Cited By

Index Terms

Porous Interfaces for Small Screen Multitasking using Finger Identification
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Investigating the feasibility of finger identification on capacitive touchscreens using deep learning
IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Touchscreens enable intuitive mobile interaction. However, touch input is limited to 2D touch locations which makes it challenging to provide shortcuts and secondary actions similar to hardware keyboards and mice. Previous work presented a wide range of ...
Touch&Screen: widget collection for large screens controlled through smartphones
MUM '16: Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia

We present Touch&Screen, a wide set of interaction techniques for the remote control of widgets (menu, lists, videos, maps etc.) for large screens through smartphones. After presenting the design of these widgets and the related control interfaces of ...
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

The natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

October 2016

908 pages

ISBN:9781450341899

DOI:10.1145/2984511

General Chairs:
Jun Rekimoto
The University of Tokyo
,
Takeo Igarashi
The University of Tokyo
,
Program Chairs:
Jacob O. Wobbrock
University of Washington
,
Daniel Avrahami
FXPAL

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada

Conference

UIST '16

Sponsor:

UIST '16: The 29th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2016

Tokyo, Japan

Acceptance Rates

UIST '16 Paper Acceptance Rate 79 of 384 submissions, 21%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
704
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lu YDing DWang RXue GKostakos VKay JHoang T(2024)HCMG: Human-Capacitance based Micro Gesture for VR/ARCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678386(766-770)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678386
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Huang ZGao CWang HDeng XLai YMa CQin SLiu YWang H(2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643559
Zhan LXiong TZhang HGuo SChen XGong JLin JQin Y(2024)TouchEditorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314547:4(1-29)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631454
Dubois EReguig IPerelman GLimousin PLouis-Pierre B(2023)Vue Radar: déporter le gestionnaire de fenêtres sur une tabletteProceedings of the 34th Conference on l'Interaction Humain-Machine10.1145/3583961.3583962(1-11)Online publication date: 3-Apr-2023
https://dl.acm.org/doi/10.1145/3583961.3583962
Yeo HWu EKim DLee JKim HOh STakagi LWoo WKoike HQuigley A(2023)OmniSense: Exploring Novel Input Sensing and Interaction Techniques on Mobile Device with an Omni-Directional CameraProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580747(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580747
Hong SHeo SLee B(2023)MaterialSense: Estimating and utilizing material properties of contact objects in multi-touch interactionInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102985172(102985)Online publication date: Apr-2023
https://doi.org/10.1016/j.ijhcs.2022.102985
Babic TReiterer HHaller M(2022)Understanding and Creating Spatial Interactions with Distant Displays Enabled by Unmodified Off-The-Shelf SmartphonesMultimodal Technologies and Interaction10.3390/mti61000946:10(94)Online publication date: 19-Oct-2022
https://doi.org/10.3390/mti6100094
Li YDo BChang SChoi J(2022)The Targeting Effects of Food Advertisements on the Video Platform: Comparing PC and Mobile Devices*Journal of Channel and Retailing10.17657/jcr.2022.1.31.5927:1(59-79)Online publication date: 31-Jan-2022
https://doi.org/10.17657/jcr.2022.1.31.59
Kim JOakley I(2022)SonarID: Using Sonar to Identify Fingers on a SmartwatchProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501935(1-10)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501935
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten