skip to main content
10.1145/2493988.2494327acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
research-article

3D from looking: using wearable gaze tracking for hands-free and feedback-free object modelling

Published: 08 September 2013 Publication History

Abstract

This paper presents a method for estimating the 3D shape of an object being observed using wearable gaze tracking. Starting from a sparse environment map generated by a simultaneous localization and mapping algorithm (SLAM), we use the gaze direction positioned in 3D to extract the model of the object under observation. By letting the user look at the object of interest, and without any feedback, the method determines 3D point-of-regards by back-projecting the user's gaze rays into the map. The 3D point-of-regards are then used as seed points for segmenting the object from captured images and the calculated silhouettes are used to estimate the 3D shape of the object. We explore methods to remove outlier gaze points that result from the user saccading to non object points and methods for reducing the error in the shape estimation. Being able to exploit gaze information in this way, enables the user of wearable gaze trackers to be able to do things as complex as object modelling in a hands-free and even feedback-free manner.

References

[1]
ASL MobileEye XG. http://www.asleyetracking.com/Site/Products/MobileEyeXG/tabid/70/Default.aspx.
[2]
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Sabine, S. SLIC Superpixels. Technical Report 149300 EPFL, June (2010), 15.
[3]
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Süsstrunk, S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence 34, 11 (Nov. 2012), 2274--82.
[4]
ASL. GazeMap, software for analysing gaze using visual mapping in 3d, 2009.
[5]
Bastian, J., Ward, B., Hill, R., van den Hengel, A., and Dick, A. Interactive modelling for AR applications. In 2010 IEEE International Symposium on Mixed and Augmented Reality, IEEE (Oct. 2010), 199--205.
[6]
Bulling, A., Ward, J., Gellersen, H., and Troster, G. Eye movement analysis for activity recognition using electrooculography. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33, 4 (2011), 741--753.
[7]
Bunnun, P., and Mayol-Cuevas, W. Outlinar: an assisted interactive model building system with reduced computational effort. In 7th IEEE and ACM International Symposium on Mixed and Augmented Reality, IEEE (September 2008).
[8]
Debevec, P. E., Taylor, C. J., and Malik, J. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM (1996), 11--20.
[9]
Felzenszwalb, P. F., and Huttenlocher, D. P. Efficient Graph-Based Image Segmentation. International Journal of Computer Vision 59, 2 (Sept. 2004), 167--181.
[10]
Graber, G., Pock, T., and Bischof, H. Online 3D reconstruction using convex optimization. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), IEEE (Nov. 2011), 708--711.
[11]
Ishiguro, Y., and Rekimoto, J. Gazecloud: A thumbnail extraction method using gaze log data for video life-log. In Wearable Computers (ISWC), 2012 16th International Symposium on (2012), 72--75.
[12]
Jacob, R. J., and Karn, K. S. Eye tracking in human--computer interaction and usability research: Ready to deliver the promises. Mind 2, 3 (2003), 4.
[13]
Jacob, R. J. K. Eye Tracking in Advanced Interface Design. In Virtual environments and advanced interface design, W. Barfield and T. A. Furness, Eds. Oxford University Press, Inc., 1995, 258--288.
[14]
Just, M. A., and Carpenter, P. A. A theory of reading: from eye fixations to comprehension. Psychological review 87, 4 (July 1980), 329--54.
[15]
Klein, G., and Murray, D. W. Parallel Tracking and Mapping for Small AR Workspaces. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, IEEE (Nov. 2007), 1--10.
[16]
Kutulakos, K., and Seitz, S. A theory of shape by space carving. Proceedings of the Seventh IEEE International Conference on Computer Vision 38, 3 (1999), 307--314 vol.1.
[17]
Land, M., Mennie, N., and Rusted, J. The roles of vision and eye movements in the control of activities of daily living. Perception 28, 11 (1999), 1311--1328.
[18]
Land, M. F. Predictable eye-head coordination during driving. Nature 359, 6393 (1992), 318--320.
[19]
Land, M. F. Eye movements and the control of actions in everyday life. Progress in retinal and eye research 25, 3 (May 2006), 296--324.
[20]
Laurentini, A. The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 2 (1994), 150--162.
[21]
McMurrough, C., Conly, C., Athitsos, V., and Makedon, F. 3D point of gaze estimation using head-mounted RGB-D cameras. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility - ASSETS '12, ACM Press (New York, New York, USA, 2012), 283.
[22]
Munn, S. M., and Pelz, J. B. 3D point-of-regard, position and head orientation from a portable monocular video-based eye tracker. Proceedings of the 2008 symposium on Eye tracking research & applications - ETRA '08 1, 212 (2008), 181.
[23]
Newcombe, R. A., Lovegrove, S. J., and Davison, A. J. DTAM: Dense tracking and mapping in real-time. In 2011 International Conference on Computer Vision, IEEE (Nov. 2011), 2320--2327.
[24]
Pan, Q., Reitmayr, G., and Drummond, T. ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition. In Proc. 20th British Machine Vision Conference (BMVC) (London, September 2009).
[25]
Piekarski, W., and Thomas, B. H. Interactive augmented reality techniques for construction at a distance of 3d geometry. In In Immersive Projection Technology Eurographics Virtual Environments (2003).
[26]
Rosten, E., and Drummond, T. Machine learning for high-speed corner detection. Computer Vision--ECCV 2006 (2006), 430--443.
[27]
Rother, C., Kolmogorov, V., and Blake, A. Grabcut: Interactive foreground extraction using iterated graph cuts. In ACM Transactions on Graphics (TOG), vol. 23, ACM (2004), 309--314.
[28]
Salvucci, D. D., and Goldberg, J. H. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the symposium on Eye tracking research & applications - ETRA '00, ACM Press (New York, New York, USA, 2000), 71--78.
[29]
Sheikh, Y. A., Khan, E. A., and Kanade, T. Mode-seeking by Medoidshifts. 2007 IEEE 11th International Conference on Computer Vision (2007), 1--8.
[30]
Stühmer, J., Gumhold, S., and Cremers, D. Real-time dense geometry from a handheld camera. In Proceedings of the 32nd DAGM conference on Pattern recognition, Springer (2010), 11--20.
[31]
Takemura, K., Kohashi, Y., Suenaga, T., Takamatsu, J., and Ogasawara, T. Estimating 3D point-of-regard and visualizing gaze trajectories under natural head movements. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA '10, vol. 1, ACM Press (New York, New York, USA, 2010), 157.

Cited By

View all
  • (2022)A Swift Gaze Estimate Method Based On The Corneal Image System2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD54268.2022.9776291(734-739)Online publication date: 4-May-2022
  • (2019)Indoor human localization based on the corneal reflection of illuminationProceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3338286.3344388(1-6)Online publication date: 1-Oct-2019
  • (2018)Towards a Symbiotic Human-Machine Depth SensorAdjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology10.1145/3266037.3266119(114-116)Online publication date: 11-Oct-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISWC '13: Proceedings of the 2013 International Symposium on Wearable Computers
September 2013
160 pages
ISBN:9781450321273
DOI:10.1145/2493988
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. algorithms
  2. experimentation

Qualifiers

  • Research-article

Conference

UbiComp '13
Sponsor:

Acceptance Rates

ISWC '13 Paper Acceptance Rate 20 of 101 submissions, 20%;
Overall Acceptance Rate 38 of 196 submissions, 19%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)A Swift Gaze Estimate Method Based On The Corneal Image System2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD54268.2022.9776291(734-739)Online publication date: 4-May-2022
  • (2019)Indoor human localization based on the corneal reflection of illuminationProceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3338286.3344388(1-6)Online publication date: 1-Oct-2019
  • (2018)Towards a Symbiotic Human-Machine Depth SensorAdjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology10.1145/3266037.3266119(114-116)Online publication date: 11-Oct-2018
  • (2017)A Survey of Wearable Devices and ChallengesIEEE Communications Surveys & Tutorials10.1109/COMST.2017.273197919:4(2573-2620)Online publication date: Dec-2018
  • (2016)Dense 3D Mapping Using Volume Reigstration from Monocular View2016 10th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS)10.1109/CISIS.2016.99(137-141)Online publication date: Jul-2016
  • (2016)Dense 3D Mapping Using Volume RegistrationNature of Computation and Communication10.1007/978-3-319-46909-6_3(22-32)Online publication date: 26-Oct-2016
  • (2015)Estimating visual attention from a head mounted IMUProceedings of the 2015 ACM International Symposium on Wearable Computers10.1145/2802083.2808394(147-150)Online publication date: 7-Sep-2015
  • (2015)Multi-User Egocentric Online System for Unsupervised Assistance on Object UsageComputer Vision - ECCV 2014 Workshops10.1007/978-3-319-16199-0_34(481-492)Online publication date: 20-Mar-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media