skip to main content
10.1145/2107596.2107597acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmumConference Proceedingsconference-collections
research-article

Augmented visualization with natural feature tracking

Published: 07 December 2011 Publication History

Abstract

Visualization systems often require large monitors or projection screens to display complex information. Even very sophisticated systems that exhibit complex user interfaces do usually not exploit advanced input and output devices. One of the reasons for that is the high cost of special hardware. This paper introduces Augmented Visualization, an interaction method for projection walls as well as monitors using affordable and widely available hardware such as mobile phones or tablets. The main technical challenge is the tracking of the users' devices without any special equipment or fiducial markers in the working area. We propose to track natural features of the display content with the built-in camera of mobile devices. Tracking the visualized scene allows pose estimation of the mobile devices with six degrees of freedom. The position and orientation information is then used for advanced interaction metaphors like magic lenses. For a group of experts who are analyzing the data in front of the same screen, a personal augmented view of the visualized scene is presented, for each user on his/her personal device. The prototype Augmented Visualization System achieves interactive frame rates and may lead to a greatly enhanced user experience. The paper discusses the design and implementation questions and illustrates potential application scenarios.

References

[1]
RFC 4629: RTP Payload Format for ITU-T Rec. H.263 Video. www.ietf.org/rfc/rfc4629.txt.
[2]
ARToolKit, (last checked on 20.08.2011). www.hitl.washington.edu/artoolkit.
[3]
GStreamer, (last checked on 20.08.2011). www.gstreamer.freedesktop.org.
[4]
Matlab Camera Calibration Toolbox, (last checked on 20.08.2011). www.vision.caltech.edu/bouguetj/calib_doc/index.html.
[5]
VLC Media Player, (last checked on 20.08.2011). www.videolan.org/vlc/.
[6]
R. Adelmann. Mobile phone based interaction with everyday products - on the go. In Proceedings of the 2007 International Conference on Next Generation Mobile Applications, Services and Technologies, pages 63--69, Washington, DC, USA, 2007. IEEE Computer Society.
[7]
R. Ballagas and M. Rohs. Mobile phones as pointing devices. In Pervasive Mobile Interaction Devices (PERMID 2005), Workshop at Pervasive 2005, pages 27--30, 2005.
[8]
H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (surf). Computer Vision and Image Understanding, 110: 346--359, June 2008.
[9]
E. A. Bier, M. C. Stone, K. Pier, W. Buxton, and T. D. DeRose. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '93, pages 73--80, New York, NY, USA, 1993. ACM.
[10]
S. Boring, M. Altendorfer, G. Broll, O. Hilliges, and A. Butz. Shoot & copy: phonecam-based information transfer from public displays onto mobile phones. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology, Mobility '07, pages 24--31, New York, NY, USA, 2007. ACM.
[11]
S. Boring, D. Baur, A. Butz, S. Gustafson, and P. Baudisch. Touchprojector: mobile interaction through video. In Proceedings of the 28th International Conference on Human Factors in Computing Systems, CHI '10, pages 2287--2296, New York, NY, USA, 2010. ACM.
[12]
S. Bruckner and E. Gröller. Volumeshop: an interactive system for direct volume illustration. In Visualization, 2005. VIS 05. IEEE, pages 671--678, 2005.
[13]
N. Cao, J. Sun, Y.-R. Lin, D. Gotz, S. Liu, and H. Qu. Facetatlas: Multifaceted visualization for rich text corpora. IEEE Transactions on Visualization and Computer Graphics, 16(6): 1172--1181, 2010.
[14]
J. Dykes, J. Wood, and A. Slingsby. Rethinking map legends with visualization. IEEE Transactions on Visualization and Computer Graphics, 16(6): 890--899, 2010.
[15]
W. Javed, B. McDonnel, and N. Elmqvist. Graphical perception of multiple time series. IEEE Transactions on Visualization and Computer Graphics, 16(6): 927--934, 2010.
[16]
S. Jeon, J. Hwang, G. J. Kim, and M. Billinghurst. Interaction with large ubiquitous displays using camera-equipped mobile phones. Personal Ubiquitous Computing, 14: 83--94, February 2010.
[17]
H. Jiang, E. Ofek, N. Moraveji, and Y. Shi. Directpointer: direct manipulation for large-display interaction using handheld cameras. In Proceedings of the SIGCHI conference on Human Factors in computing systems, CHI '06, pages 1107--1110, New York, NY, USA, 2006. ACM.
[18]
D. Kalkofen, E. Mendez, and D. Schmalstieg. Interactive focus and context visualization for augmented reality. In Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR '07, pages 1--10, Washington, DC, USA, 2007. IEEE Computer Society.
[19]
T. Quack, H. Bay, and L. Van Gool. Object recognition for the internet of things. In Proceedings of the 1st International Conference on Internet of Things, IOT'08, pages 230--246, Berlin, Heidelberg, 2008. Springer-Verlag.
[20]
M. Rohs, J. Schöning, A. Krüger, and B. Hecht. Towards real-time markerless tracking of magic lenses on paper maps. In Adjunct Proceedings of the 5th Intl. Conference on Pervasive Computing (Pervasive), Late Breaking Results, pages 69--72, 2007.
[21]
P. Roman, M. Lazarov, and A. Majumder. A scalable distributed paradigm for multi-user interaction with tiled rear projection display walls. IEEE Transactions on Visualization and Computer Graphics, 16(6): 1623--1632, 2010.
[22]
E. Rosten and T. Drummond. Fusing points and lines for high performance tracking. In IEEE International Conference on Computer Vision, volume 2, pages 1508--1511, October 2005.
[23]
E. Rosten and T. Drummond. Machine learning for high-speed corner detection. In European Conference on Computer Vision, volume 1, pages 430--443, May 2006.
[24]
J. Sanneblad and L. E. Holmquist. Ubiquitous graphics. In ACM SIGGRAPH 2005 Emerging technologies, SIGGRAPH '05, New York, NY, USA, 2005. ACM.
[25]
H. Slay, M. Phillips, R. Vernik, and B. Thomas. Interaction modes for augmented reality visualization. In Proceedings of the 2001 Asia-Pacific Symposium on Information Visualisation - Volume 9, APVis '01, pages 71--75, Darlinghurst, Australia, Australia, 2001. Australian Computer Society, Inc.
[26]
H. Slay and B. Thomas. Interaction and visualisation across multiple displays in ubiquitous computing environments. In Proceedings of the 4th International Conference on Computer graphics, Virtual Reality, Visualisation and Interaction in Africa, AFRIGRAPH '06, pages 75--84, New York, NY, USA, 2006. ACM.
[27]
B. Speckmann and K. Verbeek. Necklace maps. IEEE Transactions on Visualization and Computer Graphics, 16(6): 881--889, 2010.
[28]
M. Spindler, S. Stellmach, and R. Dachselt. Paperlens: advanced magic lens interaction above the tabletop. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS '09, pages 69--76, New York, NY, USA, 2009. ACM.
[29]
D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg. Real-time detection and tracking for augmented reality on mobile phones. IEEE Transactions on Visualization and Computer Graphics, 16(3): 355--368, 2010.
[30]
D. Wagner, D. Schmalstieg, and H. Bischof. Multiple target detection and tracking with guaranteed framerates on mobile phones. In Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR '09, pages 57--64, Washington, DC, USA, 2009. IEEE Computer Society.

Cited By

View all
  • (2019)Augmented Reality Markerless Multi-Image Outdoor Tracking System for the Historical Buildings on Parliament HillSustainability10.3390/su1116426811:16(4268)Online publication date: 7-Aug-2019
  • (2019)Augmented reality application assessment for disseminating rock artMultimedia Tools and Applications10.1007/s11042-018-6609-x78:8(10265-10286)Online publication date: 1-Apr-2019
  • (2017)Spatial interaction for the post-processing of 3D CFD datasetsProceedings of the Eurographics/IEEE VGTC Conference on Visualization: Short Papers10.2312/eurovisshort.20171136(73-77)Online publication date: 12-Jun-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
MUM '11: Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia
December 2011
242 pages
ISBN:9781450310963
DOI:10.1145/2107596
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • Tsinghua University: Tsinghua University

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 December 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. handheld augmented reality
  2. human computer interaction
  3. interactive visualization systems
  4. natural feature tracking

Qualifiers

  • Research-article

Funding Sources

Conference

MUM'11
Sponsor:
  • Tsinghua University

Acceptance Rates

MUM '11 Paper Acceptance Rate 29 of 66 submissions, 44%;
Overall Acceptance Rate 190 of 465 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Augmented Reality Markerless Multi-Image Outdoor Tracking System for the Historical Buildings on Parliament HillSustainability10.3390/su1116426811:16(4268)Online publication date: 7-Aug-2019
  • (2019)Augmented reality application assessment for disseminating rock artMultimedia Tools and Applications10.1007/s11042-018-6609-x78:8(10265-10286)Online publication date: 1-Apr-2019
  • (2017)Spatial interaction for the post-processing of 3D CFD datasetsProceedings of the Eurographics/IEEE VGTC Conference on Visualization: Short Papers10.2312/eurovisshort.20171136(73-77)Online publication date: 12-Jun-2017
  • (2017)Interactive Lenses for VisualizationComputer Graphics Forum10.1111/cgf.1287136:6(173-200)Online publication date: 1-Sep-2017
  • (2014)Collaborative Information Visualization Using a Multi-projection System and Mobile Devices2014 18th International Conference on Information Visualisation10.1109/IV.2014.23(71-77)Online publication date: Jul-2014
  • (2013)Visdom MobileProceedings of the 29th Spring Conference on Computer Graphics10.1145/2508244.2508257(99-106)Online publication date: 1-May-2013
  • (2013)A review of display projection system: Its problems and challenges2013 IEEE Conference on Open Systems (ICOS)10.1109/ICOS.2013.6735075(203-208)Online publication date: Dec-2013

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media