short-paper

TiPoint: detecting fingertip for mid-air interaction on computational resource constrained smartglasses

Authors:

Lik Hang Lee,

Tristan Braud,

Farshid Hassani Bijarbooneh,

Pan HuiAuthors Info & Claims

ISWC '19: Proceedings of the 2019 ACM International Symposium on Wearable Computers

Pages 118 - 122

https://doi.org/10.1145/3341163.3347723

Published: 09 September 2019 Publication History

Get Access

Abstract

Smartglasses mostly rely on hardware interfaces such as touch-pad and buttons, which are often cumbersome and counter-intuitive to use. Furthermore, smartglasses feature cheap and low-power hardware preventing the use of advanced pointing techniques. To overcome these issues, we introduce TiPoint, a freehand mid-air interaction technique. TiPoint uses the monocular camera embedded in smartglasses to detect the user's hand, enabling intuitive and non-intrusive interaction. We introduce a light-weight algorithm for fingertip detection, which is especially suited for the limited hardware specifications and the short battery life time of smartglasses. Our evaluation shows that TiPoint as a mid-air non-intrusive interface delivers a better experience for users and smart glasses interactions, with users completing typical tasks 1.82 times faster than when using the original hardware.

References

[1]

M. Abramowitz and I.A. Stegun. 1972. Handbook of mathematical functions: with formulas, graphs, and mathematical tables. Courier Dover Publications (55).

Google Scholar

[2]

W. Buxton and B. Myers. 1986. A Study in Two-handed Input. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (CHI '86).

Digital Library

Google Scholar

[3]

Georgiana Simion et al. 2016a. Fingertip-based real time tracking and gesture recognition for natural user interfaces. 13 (01 2016), 189--204.

Google Scholar

[4]

P. Hamette et al. 2002. FingerMouse: A Wearable Hand Tracking System. (01 2002).

Google Scholar

[5]

R. Gang et al. 2013. 3D selection with freehand gesture. Computers & Graphics 37, 3 (2013), 101--120.

Digital Library

Google Scholar

[6]

Wang et al. 2015. PalmType: Using Palms As Keyboards for Smart Glasses. In Proc. of the 17th Int. Conf. on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, 153--160.

Digital Library

Google Scholar

[7]

Yichao Huang et al. 2016b. A Pointing Gesture Based Egocentric Interaction System: Dataset, Approach and Application. (2016).

Google Scholar

[8]

Y.-T. Hsieh et al. 2016c. Designing a Willing-to-Use-in-Public Hand Gestural Interaction Technique for Smart Glasses. In Proc. of the 2016 CHI Conf. on Human Factors in Computing Systems (CHI '16).

Digital Library

Google Scholar

[9]

Z. Huang, W. Li, and P. Hui. 2015. Ubii: Towards Seamless Interaction Between Digital and Physical Worlds. In Proc. of the 23rd ACM Int. Conf. on Multimedia (MM '15).

Digital Library

Google Scholar

[10]

S. Lawrence and W. Brett. 2013. Comparison of Gestural, Touch, and Mouse Interaction with Fitts' Law. In Proc. of the 25th Australian Computer-Human Interaction Conf.: Augmentation, Application, Innovation, Collaboration (OzCHI '13). ACM, 119--122.

Digital Library

Google Scholar

[11]

T. Lee and T. Hollerer. 2007. Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking. In 2007 11th IEEE Int. Sym. on Wearable Computers. 83--90.

Digital Library

Google Scholar

[12]

T. E. Starner. 2002. Attention, memory, and wearable interfaces. IEEE Pervasive Computing 1, 4 (Oct 2002).

Digital Library

Google Scholar

Cited By

View all

Campos TCastello MDamasceno EValentim N(2025)An Updated Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holographic SolutionsJournal on Interactive Systems10.5753/jis.2025.469416:1(172-198)Online publication date: 1-Jan-2025
https://doi.org/10.5753/jis.2025.4694
Lee LHosio SBraud TZhou P(2024)A Roadmap Toward Metaversity: Recent Developments and Perspectives in EducationApplication of the Metaverse in Education10.1007/978-981-97-1298-4_5(73-95)Online publication date: 8-May-2024
https://doi.org/10.1007/978-981-97-1298-4_5
Campos TDamasceno EValentim N(2023)Usability and User Experience Evaluation of Touchable Holographic Solutions: A Systematic Mapping StudyProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638071(1-13)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638071
Show More Cited By

Recommendations

Visual touchpad: a two-handed gestural input device
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

This paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar ...
Touch+Finger: Extending Touch-based User Interface Capabilities with "Idle" Finger Gestures in the Air
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

In this paper, we present Touch+Finger, a new interaction technique that augments touch input with multi-finger gestures for rich and expressive interaction. The main idea is that while one finger is engaged in a touch event, a user can leverage the ...
T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation
SUI '14: Proceedings of the 2nd ACM symposium on Spatial user interaction

T(ether) is a spatially-aware display system for multi-user, collaborative manipulation and animation of virtual 3D objects. The handheld display acts as a window into virtual reality, providing users with a perspective view of 3D data. T(ether) tracks ...

Comments

Information & Contributors

Information

Published In

ISWC '19: Proceedings of the 2019 ACM International Symposium on Wearable Computers

September 2019

355 pages

ISBN:9781450368704

DOI:10.1145/3341163

General Chairs:
Katayoun Farrahi
University Of Southampton
,
Robert Harle
Cambridge University
,
Nicholas Lane
University Of Oxford And Samsung Ai

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

UbiComp '19

Sponsor:

UbiComp '19: The 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 9 - 13, 2019

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 38 of 196 submissions, 19%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
294
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)8

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Campos TCastello MDamasceno EValentim N(2025)An Updated Systematic Mapping Study on Usability and User Experience Evaluation of Touchable Holographic SolutionsJournal on Interactive Systems10.5753/jis.2025.469416:1(172-198)Online publication date: 1-Jan-2025
https://doi.org/10.5753/jis.2025.4694
Lee LHosio SBraud TZhou P(2024)A Roadmap Toward Metaversity: Recent Developments and Perspectives in EducationApplication of the Metaverse in Education10.1007/978-981-97-1298-4_5(73-95)Online publication date: 8-May-2024
https://doi.org/10.1007/978-981-97-1298-4_5
Campos TDamasceno EValentim N(2023)Usability and User Experience Evaluation of Touchable Holographic Solutions: A Systematic Mapping StudyProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638071(1-13)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638071
Li ZZhang XWu C(2023)RingVKB: A Ring-Shaped Virtual Keyboard Using Low-Cost IMUProceedings of the ACM on Human-Computer Interaction10.1145/36042677:MHCI(1-20)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604267
Shatilov KKwon YLee LChatzopoulos DHui P(2023)MyoKey: Inertial Motion Sensing and Gesture-Based QWERTY Keyboard for Extended RealitiesIEEE Transactions on Mobile Computing10.1109/TMC.2022.315693922:8(4807-4821)Online publication date: 1-Aug-2023
https://doi.org/10.1109/TMC.2022.3156939
Lee LBraud THui P(2023)Embodied Interaction on Constrained Interfaces for Augmented RealitySpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_10(239-271)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_10
Lam KLee LHui PMurray NSimon GFarias MViola IMontagud M(2022)3DeformRProceedings of the 13th ACM Multimedia Systems Conference10.1145/3524273.3528180(52-61)Online publication date: 14-Jun-2022
https://dl.acm.org/doi/10.1145/3524273.3528180
Zhang HYin YXie LGu TYou MLu S(2022)DynaKey: Dynamic Keystroke Tracking Using a Head-Mounted Camera DeviceIEEE Internet of Things Journal10.1109/JIOT.2021.31142249:9(6563-6577)Online publication date: 1-May-2022
https://doi.org/10.1109/JIOT.2021.3114224
Kim DChoi Y(2021)Applications of Smart Glasses in Applied Sciences: A Systematic ReviewApplied Sciences10.3390/app1111495611:11(4956)Online publication date: 27-May-2021
https://doi.org/10.3390/app11114956
Lam KLee LHui PShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web BrowserProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475413(2447-2455)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475413
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Visual touchpad: a two-handed gestural input device

Touch+Finger: Extending Touch-based User Interface Capabilities with "Idle" Finger Gestures in the Air

T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations