skip to main content
10.1145/2999508.2999530acmconferencesArticle/Chapter ViewAbstractPublication Pagessiggraph-asiaConference Proceedingsconference-collections
research-article

ubiGaze: ubiquitous augmented reality messaging using gaze gestures

Published: 28 November 2016 Publication History

Abstract

We describe ubiGaze, a novel wearable ubiquitous method to augment any real-world object with invisible messages through gaze gestures that lock the message into the object. This enables a context and location dependent messaging service, which users can utilize discreetly and effortlessly. Further, gaze gestures can be used as an authentication method, even when the augmented object is publicly known. We developed a prototype using two wearable devices: a Pupil eye tracker equipped with a scene camera and a Sony Smartwatch 3. The eye tracker follows the users' gaze, the scene camera captures distinct features from the selected real-world object, and the smartwatch provides both input and output modalities for selecting and displaying messages. We describe the concept, design, and implementation of our real-world system. Finally, we discuss research implications and address future work.

Supplementary Material

ZIP File (a11-bace.zip)
Supplemental files.

References

[1]
Azuma, R. T. 1997. A survey of augmented reality. Presence: Teleoperators and Virtual Environments 6, 4 (Aug.), 355--385.
[2]
Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. 2008. Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding 110, 3 (June), 346--359.
[3]
Billinghurst, M., Clark, A., and Lee, G. 2015. A survey of augmented reality. Foundations and Trends in Human-Computer Interaction 8, 2-3 (Mar.), 73--272.
[4]
De Luca, A., Weiss, R., and Drewes, H. 2007. Evaluation of eye-gaze interaction methods for security enhanced pin-entry. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces, ACM, OZCHI '07, 199--202.
[5]
Drewes, H., and Schmidt, A. 2007. Interacting with the computer using gaze gestures. In Proceedings of the 11th IFIP TC 13 International Conference on Human-computer Interaction - Volume Part II, Springer-Verlag, INTERACT'07, 475--488.
[6]
Drewes, H., De Luca, A., and Schmidt, A. 2007. Eye-gaze interaction for mobile phones. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology, ACM, Mobility '07, 364--371.
[7]
Fedosov, A., Elhart, I., Niforatos, E., North, A., and Langheinrich, M. 2016. SkiAR: Wearable augmented reality system for sharing personalized content on ski resort maps. In Proceedings of the 7th Augmented Human International Conference 2016, ACM, AH '16, 46:1--46:2.
[8]
Hyrskykari, A., Istance, H., and Vickers, S. 2012. Gaze gestures or dwell-based interaction? In Proceedings of the Symposium on Eye Tracking Research & Applications, ACM, ETRA '12, 229--232.
[9]
Istance, H., Hyrskykari, A., Immonen, L., Mansikkamaa, S., and Vickers, S. 2010. Designing gaze gestures for gaming: An investigation of performance. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, ACM, ETRA '10, 323--330.
[10]
Kangas, J., Akkil, D., Rantala, J., Isokoski, P., Majaranta, P., and Raisamo, R. 2014. Gaze gestures and haptic feedback in mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, CHI '14, 435--438.
[11]
Mayer, S., and Sörös, G. 2014. User interface beaming - seamless interaction with smart things using personal wearable computers. In Proceedings of the 11th International Conference on Wearable and Implantable Body Sensor Networks (BSN 2014), 46--49.
[12]
Mistry, P., and Maes, P. 2009. SixthSense: A wearable gestural interface. In ACM SIGGRAPH ASIA 2009 Sketches, ACM, SIGGRAPH ASIA '09, 11:1--11:1.
[13]
Mistry, P., Maes, P., and Chang, L. 2009. WUW - wear ur world: A wearable gestural interface. In CHI '09 Extended Abstracts on Human Factors in Computing Systems, ACM, CHI EA '09, 4111--4116.
[14]
Møllenbach, E., Hansen, J. P., Lillholm, M., and Gale, A. G. 2009. Single stroke gaze gestures. In CHI '09 Extended Abstracts on Human Factors in Computing Systems, ACM, CHI EA '09, 4555--4560.
[15]
Møllenbach, E., Lillholm, M., Gail, A., and Hansen, J. P. 2010. Single gaze gestures. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, ACM, ETRA '10, 177--180.
[16]
Nassani, A., Bai, H., Lee, G., and Billinghurst, M. 2015. Tag it!: AR annotation using wearable sensors. In SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications, ACM, SA '15, 12:1--12:4.
[17]
Nuernberger, B., Lien, K. C., Höllerer, T., and Turk, M. 2016. Interpreting 2D gesture annotations in 3D augmented reality. In 2016 IEEE Symposium on 3D User Interfaces (3DUI), 149--158.
[18]
Quintana, E., and Favela, J. 2013. Augmented reality annotations to assist persons with Alzheimers and their caregivers. Personal and Ubiquitous Computing 17, 6 (Aug.), 1105--1116.
[19]
Wobbrock, J. O., Rubinstein, J., Sawyer, M. W., and Duchowski, A. T. 2008. Longitudinal evaluation of discrete consecutive gaze gestures for text entry. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications, ACM, ETRA '08, 11--18.
[20]
Yilmaz, A., Javed, O., and Shah, M. 2006. Object tracking: A survey. ACM Computing Surveys 38, 4 (Dec.).

Cited By

View all
  • (2024)Between Wearable and Spatial Computing: Exploring Four Interaction Techniques at the Intersection of Smartwatches and Head-mounted DisplaysProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656365(1-7)Online publication date: 4-Jun-2024
  • (2023)Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social ConnectionsProceedings of the ACM on Human-Computer Interaction10.1145/36102087:CSCW2(1-29)Online publication date: 4-Oct-2023
  • (2023)Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581461(1-14)Online publication date: 19-Apr-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SA '16: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications
November 2016
76 pages
ISBN:9781450345514
DOI:10.1145/2999508
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. augmented reality
  2. eye tracking
  3. gaze gestures
  4. gesture interaction
  5. messaging

Qualifiers

  • Research-article

Conference

SA '16
Sponsor:
SA '16: SIGGRAPH Asia 2016
December 5 - 8, 2016
Macau

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)49
  • Downloads (Last 6 weeks)3
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Between Wearable and Spatial Computing: Exploring Four Interaction Techniques at the Intersection of Smartwatches and Head-mounted DisplaysProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656365(1-7)Online publication date: 4-Jun-2024
  • (2023)Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social ConnectionsProceedings of the ACM on Human-Computer Interaction10.1145/36102087:CSCW2(1-29)Online publication date: 4-Oct-2023
  • (2023)Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581461(1-14)Online publication date: 19-Apr-2023
  • (2023)Cartographic perspectives on spatial and thematic levels of detail in augmented reality: a review of existing approachesInternational Journal of Cartography10.1080/23729333.2023.22285929:2(373-391)Online publication date: 6-Jul-2023
  • (2022)Scalable Extended Reality: A Future Research AgendaBig Data and Cognitive Computing10.3390/bdcc60100126:1(12)Online publication date: 26-Jan-2022
  • (2022)NotiBike: Assessing Target Selection Techniques for Cyclist Notifications in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/35467326:MHCI(1-24)Online publication date: 20-Sep-2022
  • (2022)The Eye in Extended Reality: A Survey on Gaze Interaction and Eye Tracking in Head-worn Extended RealityACM Computing Surveys10.1145/349120755:3(1-39)Online publication date: 25-Mar-2022
  • (2021)Gaze+Hold: Eyes-only Direct Manipulation with Continuous Gaze Modulated by Closure of One EyeACM Symposium on Eye Tracking Research and Applications10.1145/3448017.3457381(1-12)Online publication date: 25-May-2021
  • (2021)Ubiquitous Interactions for Heads-Up Computing: Understanding Users’ Preferences for Subtle Interaction Techniques in Everyday SettingsProceedings of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447526.3472035(1-15)Online publication date: 27-Sep-2021
  • (2021)Interaction With Gaze, Gesture, and Speech in a Flexibly Configurable Augmented Reality SystemIEEE Transactions on Human-Machine Systems10.1109/THMS.2021.309797351:5(524-534)Online publication date: Oct-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media