research-article

Declarative AR and image processing on the web with Xflow

Authors:

Dmitri Rubinstein,

Farshad Einabadi,

Stephan Herhut,

Philipp SlusallekAuthors Info & Claims

Web3D '13: Proceedings of the 18th International Conference on 3D Web Technology

Pages 157 - 165

https://doi.org/10.1145/2466533.2466544

Published: 20 June 2013 Publication History

Abstract

Recently, modern Web browser became capable of supporting powerful, interactive 3D graphics both via the low-level, imperative API of WebGL as well as via a high-level, declarative approach like XML3D. The obvious next step (particularly with respect to mobile platforms) is to combine video from the real world with matched virtual content -- Augmented or Mixed Reality (AR/MR). However, AR requires extensive image or video processing, feature detection and tracking, and applying the results to 3D rendering -- all of which is hard to implement in a Web context.

In this paper we present a novel approach that encapsulates low-level image-processing and AR operations into re-usable high-level XML3D/Xflow components that are part of the HTML-5 DOM. A Web developer can then easily and flexibly arrange these components into (possibly complex) processing flow-graphs without having to worry about the internal computations and the structure of these modules. Our extended Xflow implementation automatically optimizes, schedules, and synchronizes the processing of the flow graph(s) in the context of real-time 3D rendering. Finally, we provide an integration model that greatly simplifies building AR applications for the browser.

We demonstrate this with several simple AR and image processing applications using a polyfill implementation working in all modern browsers and evaluate the performance. Finally, we show how the declarative framework can be optimized with respect to performance and usability using parallelization with Web Workers and RiverTrail.

Supplementary Material

JPG File (p157-klein.jpg)

Download
18.62 KB

AVI File (p157-klein.avi)

Download
5.45 MB

References

[1]

Behr, J., Eschler, P., Jung, Y., and Zöllner, M. 2009. X3DOM: a DOM-based HTML5/X3D integration model. In Proceedings of the 14th International Conference on 3D Web Technology, ACM, New York, NY, USA, Web3D '09, 127--135.

Digital Library

[2]

Heikkinen, I., 2012. Writing augmenged reality applications using JSARToolKit. URL: http://www.html5rocks.com/en/tutorials/webgl/jsartoolkit webrtc/, February.

[3]

Herhut, S., Hudson, R. L., Shpeisman, T., and Sreeram, J. 2012. Parallel programming for the web. In Proceedings of the 4th USENIX conference on Hot Topics in Parallelism, Hot-Par'12, USENIX Association.

Digital Library

[4]

Hudson, R. L., and Herhut, S., 2013. Parallel EcmaScript (River Trail) API. http://wiki.ecmascript.org/doku.php?id=strawman:data_parallelism, February.

[5]

Kato, H., Billinghurst, M., Blanding, R., and May, R. 1999. ARToolKit. Technical report, Hiroshima City University, December.

[6]

Khronos Group, 2011. WebGL - OpenGL ES 2.0 for the Web. URL: http://www.khronos.org/webgl, May.

[7]

Klein, F., Sons, K., Rubinstein, D., Byelozyorov, S., John, S., and Slusallek, P. 2012. Xflow - declarative data processing for the web. In Proceedings of the 17th International Conference on Web 3D Technology.

Digital Library

[8]

OpenCV Foundation, 2012. Open Source Computer Vision (OpenCV). http://opencv.org/.

[9]

Otsu, N. 1979. A threshold selection method from gray-level histograms. Systems, Man and Cybernetics, IEEE Transactions on 9, 1 (January), 62--66.

[10]

Salga, A., Hodgin, D., Sobiepanek, A., Downe, S., Medel, M., and Leung, C. 2011. Processing.js: sketching with <canvas>. In SIGGRAPH Talks, ACM, M. Elendt, Ed., 15.

Digital Library

[11]

Sons, K., Klein, F., Rubinstein, D., Byelozyorov, S., and Slusallek, P. 2010. XML3D: Interactive 3D Graphics for the Web. In Proceedings of the 15th International Conference on Web 3D Technology, ACM, New York, NY, USA, Web3D '10, 175--184.

Digital Library

[12]

Sons, K., Schlinkmann, C., Klein, F., Rubinstein, D., and Slusallek, P. 2013. xml3d.js: Architecture of a polyfill implementation of xml3d. In SEARIS.

[13]

W3C, 2006. The Canvas 2D API 1.0 Specification -- Editor's Draft. http://dev.w3.org/2006/canvas-api/canvas-2d-api.html.

[14]

W3C, 2012. CSS Transforms -- Working Draft. http://www.w3.org/TR/css3-3d-transforms/, September.

[15]

W3C, 2012. DOM4 Specification -- Working Draft. https://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html, December.

[16]

W3C, 2012. Filter Effects 1.0 -- Working Draft. http://www.w3.org/TR/filter-effects/, October.

[17]

W3C, 2012. Media Capture and Streams -- Editor's Draft. http://dev.w3.org/2011/webrtc/editor/getusermedia.html, December.

[18]

Web3D Consortium, 1997. ISO/IEC 14772-1:1997, The Virtual Reality Modeling Language (VRML). http://www.web3d.org/x3d/specifications/vrml/ISO-IEC-14772-VRML97/.

[19]

WHATWG, 2013. HTML Living Standard - The Canvas Element. http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#imagedata, February.

Cited By

Lam KLee LHui PShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web BrowserProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475413(2447-2455)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475413
Lam KHang Lee LBraud THui P(2019)M2A: A Framework for Visualizing Information from Mobile Web to Mobile Augmented Reality2019 IEEE International Conference on Pervasive Computing and Communications (PerCom10.1109/PERCOM.2019.8767388(1-10)Online publication date: Mar-2019
https://doi.org/10.1109/PERCOM.2019.8767388
Al-Zoube M(2017)Web-Based Augmented Reality with Natural Feature Tracking and Advanced Rendering2017 International Conference on New Trends in Computing Sciences (ICTCS)10.1109/ICTCS.2017.44(320-326)Online publication date: Oct-2017
https://doi.org/10.1109/ICTCS.2017.44
Show More Cited By

Index Terms

Declarative AR and image processing on the web with Xflow
1. Computing methodologies
  1. Computer graphics
    1. Animation
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques

Recommendations

Xflow: declarative data processing for the web
Web3D '12: Proceedings of the 17th International Conference on 3D Web Technology

The Web evolved from a simple information medium to an interactive application platform featuring advanced 2D layouts, videos, and audio. At the same time, support for hardware-accelerated 3D graphics improves continuously even among mobile devices. ...
XML3D and Xflow: Combining Declarative 3D for the Web with Generic Data Flows

Researchers have combined XML3D, which provides declarative, interactive 3D scene descriptions based on HTML5, with Xflow, a language for declarative, high-performance data processing. The result lets Web developers combine a 3D scene graph with data ...
Configurable instances of 3D models for declarative 3D in the web
Web3D '14: Proceedings of the 19th International ACM Conference on 3D Web Technologies

The Declarative 3D for the Web initiative by the W3C [W3C 2011] connects 3D content to the Web document, intertwining it with other Web technologies known to millions of Web developers. The goal is to make 3D on the Web more accessible compared to low-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Web3D '13: Proceedings of the 18th International Conference on 3D Web Technology

June 2013

220 pages

ISBN:9781450321334

DOI:10.1145/2466533

General Chairs:
Jorge Posada
Vicomtech, Spain
,
Don Brutzman
Naval Postgraduate School
,
Program Chairs:
Denis Graèanin
Virginia Tech
,
Byounghyun Yoo
KIST, Korea
,
David Oyarzun
Vicomtech-IK4, Spain

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

Web3D '13

Sponsor:

SIGGRAPH

Web3D '13: The 18th International Conference on Web3D Technology

June 20 - 22, 2013

San Sebastian, Spain

Acceptance Rates

Overall Acceptance Rate 27 of 71 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
246
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lam KLee LHui PShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web BrowserProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475413(2447-2455)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475413
Lam KHang Lee LBraud THui P(2019)M2A: A Framework for Visualizing Information from Mobile Web to Mobile Augmented Reality2019 IEEE International Conference on Pervasive Computing and Communications (PerCom10.1109/PERCOM.2019.8767388(1-10)Online publication date: Mar-2019
https://doi.org/10.1109/PERCOM.2019.8767388
Al-Zoube M(2017)Web-Based Augmented Reality with Natural Feature Tracking and Advanced Rendering2017 International Conference on New Trends in Computing Sciences (ICTCS)10.1109/ICTCS.2017.44(320-326)Online publication date: Oct-2017
https://doi.org/10.1109/ICTCS.2017.44
Karhu AHeikkinen AKoskela T(2014)Towards Augmented Reality Applications in a Mobile Web ContextProceedings of the 2014 Eighth International Conference on Next Generation Mobile Apps, Services and Technologies10.1109/NGMAST.2014.36(1-6)Online publication date: 10-Sep-2014
https://dl.acm.org/doi/10.1109/NGMAST.2014.36
Leppänen THeikkinen AKarhu AHarjula ERiekki JKoskela T(2014)Augmented Reality Web Applications with Mobile Agents in the Internet of ThingsProceedings of the 2014 Eighth International Conference on Next Generation Mobile Apps, Services and Technologies10.1109/NGMAST.2014.24(54-59)Online publication date: 10-Sep-2014
https://dl.acm.org/doi/10.1109/NGMAST.2014.24

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten