Gábor Sörös
ABSTRACT
Visualization systems often require large monitors or projection screens to display complex information. Even very sophisticated systems that exhibit complex user interfaces do usually not exploit advanced input and output devices. One of the reasons for that is the high cost of special hardware. This paper introduces Augmented Visualization, an interaction method for projection walls as well as monitors using affordable and widely available hardware such as mobile phones or tablets. The main technical challenge is the tracking of the users' devices without any special equipment or fiducial markers in the working area. We propose to track natural features of the display content with the built-in camera of mobile devices. Tracking the visualized scene allows pose estimation of the mobile devices with six degrees of freedom. The position and orientation information is then used for advanced interaction metaphors like magic lenses. For a group of experts who are analyzing the data in front of the same screen, a personal augmented view of the visualized scene is presented, for each user on his/her personal device. The prototype Augmented Visualization System achieves interactive frame rates and may lead to a greatly enhanced user experience. The paper discusses the design and implementation questions and illustrates potential application scenarios.
- RFC 4629: RTP Payload Format for ITU-T Rec. H.263 Video. www.ietf.org/rfc/rfc4629.txt.Google Scholar
- ARToolKit, (last checked on 20.08.2011). www.hitl.washington.edu/artoolkit.Google Scholar
- GStreamer, (last checked on 20.08.2011). www.gstreamer.freedesktop.org.Google Scholar
- Matlab Camera Calibration Toolbox, (last checked on 20.08.2011). www.vision.caltech.edu/bouguetj/calib_doc/index.html.Google Scholar
- VLC Media Player, (last checked on 20.08.2011). www.videolan.org/vlc/.Google Scholar
- R. Adelmann. Mobile phone based interaction with everyday products - on the go. In Proceedings of the 2007 International Conference on Next Generation Mobile Applications, Services and Technologies, pages 63--69, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- R. Ballagas and M. Rohs. Mobile phones as pointing devices. In Pervasive Mobile Interaction Devices (PERMID 2005), Workshop at Pervasive 2005, pages 27--30, 2005.Google Scholar
- H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (surf). Computer Vision and Image Understanding, 110: 346--359, June 2008. Google ScholarDigital Library
- E. A. Bier, M. C. Stone, K. Pier, W. Buxton, and T. D. DeRose. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '93, pages 73--80, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- S. Boring, M. Altendorfer, G. Broll, O. Hilliges, and A. Butz. Shoot & copy: phonecam-based information transfer from public displays onto mobile phones. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology, Mobility '07, pages 24--31, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- S. Boring, D. Baur, A. Butz, S. Gustafson, and P. Baudisch. Touchprojector: mobile interaction through video. In Proceedings of the 28th International Conference on Human Factors in Computing Systems, CHI '10, pages 2287--2296, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- S. Bruckner and E. Gröller. Volumeshop: an interactive system for direct volume illustration. In Visualization, 2005. VIS 05. IEEE, pages 671--678, 2005.Google ScholarCross Ref
- N. Cao, J. Sun, Y.-R. Lin, D. Gotz, S. Liu, and H. Qu. Facetatlas: Multifaceted visualization for rich text corpora. IEEE Transactions on Visualization and Computer Graphics, 16(6): 1172--1181, 2010. Google ScholarDigital Library
- J. Dykes, J. Wood, and A. Slingsby. Rethinking map legends with visualization. IEEE Transactions on Visualization and Computer Graphics, 16(6): 890--899, 2010. Google ScholarDigital Library
- W. Javed, B. McDonnel, and N. Elmqvist. Graphical perception of multiple time series. IEEE Transactions on Visualization and Computer Graphics, 16(6): 927--934, 2010. Google ScholarDigital Library
- S. Jeon, J. Hwang, G. J. Kim, and M. Billinghurst. Interaction with large ubiquitous displays using camera-equipped mobile phones. Personal Ubiquitous Computing, 14: 83--94, February 2010. Google ScholarDigital Library
- H. Jiang, E. Ofek, N. Moraveji, and Y. Shi. Directpointer: direct manipulation for large-display interaction using handheld cameras. In Proceedings of the SIGCHI conference on Human Factors in computing systems, CHI '06, pages 1107--1110, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- D. Kalkofen, E. Mendez, and D. Schmalstieg. Interactive focus and context visualization for augmented reality. In Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR '07, pages 1--10, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarDigital Library
- T. Quack, H. Bay, and L. Van Gool. Object recognition for the internet of things. In Proceedings of the 1st International Conference on Internet of Things, IOT'08, pages 230--246, Berlin, Heidelberg, 2008. Springer-Verlag. Google ScholarDigital Library
- M. Rohs, J. Schöning, A. Krüger, and B. Hecht. Towards real-time markerless tracking of magic lenses on paper maps. In Adjunct Proceedings of the 5th Intl. Conference on Pervasive Computing (Pervasive), Late Breaking Results, pages 69--72, 2007.Google Scholar
- P. Roman, M. Lazarov, and A. Majumder. A scalable distributed paradigm for multi-user interaction with tiled rear projection display walls. IEEE Transactions on Visualization and Computer Graphics, 16(6): 1623--1632, 2010. Google ScholarDigital Library
- E. Rosten and T. Drummond. Fusing points and lines for high performance tracking. In IEEE International Conference on Computer Vision, volume 2, pages 1508--1511, October 2005. Google ScholarDigital Library
- E. Rosten and T. Drummond. Machine learning for high-speed corner detection. In European Conference on Computer Vision, volume 1, pages 430--443, May 2006. Google ScholarDigital Library
- J. Sanneblad and L. E. Holmquist. Ubiquitous graphics. In ACM SIGGRAPH 2005 Emerging technologies, SIGGRAPH '05, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- H. Slay, M. Phillips, R. Vernik, and B. Thomas. Interaction modes for augmented reality visualization. In Proceedings of the 2001 Asia-Pacific Symposium on Information Visualisation - Volume 9, APVis '01, pages 71--75, Darlinghurst, Australia, Australia, 2001. Australian Computer Society, Inc. Google ScholarDigital Library
- H. Slay and B. Thomas. Interaction and visualisation across multiple displays in ubiquitous computing environments. In Proceedings of the 4th International Conference on Computer graphics, Virtual Reality, Visualisation and Interaction in Africa, AFRIGRAPH '06, pages 75--84, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- B. Speckmann and K. Verbeek. Necklace maps. IEEE Transactions on Visualization and Computer Graphics, 16(6): 881--889, 2010. Google ScholarDigital Library
- M. Spindler, S. Stellmach, and R. Dachselt. Paperlens: advanced magic lens interaction above the tabletop. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS '09, pages 69--76, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg. Real-time detection and tracking for augmented reality on mobile phones. IEEE Transactions on Visualization and Computer Graphics, 16(3): 355--368, 2010. Google ScholarDigital Library
- D. Wagner, D. Schmalstieg, and H. Bischof. Multiple target detection and tracking with guaranteed framerates on mobile phones. In Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR '09, pages 57--64, Washington, DC, USA, 2009. IEEE Computer Society. Google ScholarDigital Library
Index Terms
- Augmented visualization with natural feature tracking
Recommendations
A hybrid pose tracking approach for handheld augmented reality
ICDSC '15: Proceedings of the 9th International Conference on Distributed Smart CamerasWith the rapid advances in mobile computing, handheld Augmented Reality draws increasing attention. Pose tracking of handheld devices is of fundamental importance to register virtual information with the real world and is still a crucial challenge. In ...
Efficient pose tracking from natural features in standard web browsers
Web3D '18: Proceedings of the 23rd International ACM Conference on 3D Web TechnologyComputer Vision-based natural feature tracking is at the core of modern Augmented Reality applications. Still, Web-based Augmented Reality typically relies on location-based sensing (using GPS and orientation sensors) or marker-based approaches to solve ...
Comparative performance evaluation of CAR systems based on mobile phones and feature tracking
Collaborative Augmented Reality (CAR) systems based on mobile phones have experienced a huge expansion last years, since the hardware features of most mobile phones provide excellent multimedia services and wireless network capabilities. In previous ...
Comments