research-article

DejaVu: integrated support for developing interactive camera-based programs

Authors:

Xiang CaoAuthors Info & Claims

UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

Pages 189 - 196

https://doi.org/10.1145/2380116.2380142

Published: 07 October 2012 Publication History

Abstract

The increasing popularity of interactive camera-based programs highlights the inadequacies of conventional IDEs in developing these programs given their distinctive attributes and workflows. We present DejaVu, an IDE enhancement that eases the development of these programs by enabling programmers to visually and continuously monitor program data in consistency with the frame-based pipeline of computer-vision programs; and to easily record, review, and reprocess temporal data to iteratively improve the processing of non-reproducible camera input. DejaVu was positively received by three experienced programmers of interactive camera-based programs in our preliminary user trial.

Supplementary Material

JPG File (paper_0211-file3.jpg)

Download
24.93 KB

suppl.mov (paper_0211-file3.mp4)

Supplemental video

Download
53.34 MB

References

[1]

Balzer, R. (1969). EXDAMS: extensible debugging and monitoring system. Spring joint computer conference, p. 567--580.

Digital Library

[2]

Bradski, G. (2000). The OpenCV Library. Dr. Dobb's Journal November 2000, Computer Security.

[3]

Cao, X., Balakrishnan, R. (2003). VisionWand: interaction techniques for large displays using a passive wand tracked in 3D. UIST, p. 173--182.

Digital Library

[4]

Cardenas, T., Bastea-Forte, M., Ricciardi, A., Hartmann, B., Klemmer, S. (2008). Testing physical computing prototypes through time-shifted & simulated input traces. UIST adjunct proceedings.

[5]

Diaz-Marino, R., Greenberg, S. (2006). CAMBIENCE: A Video-Driven Sonic Ecology for Media Spaces. Video Proceedings of CSCW.

[6]

Edwards, J. (2005). Subtext: Uncovering the simplicity of programming. OOPSLA, p. 505--518.

Digital Library

[7]

Fails, J., Olsen, D. (2003). A design tool for camera-based interaction. CHI, p. 449--456.

Digital Library

[8]

Geels, D., Altekar, G., Shenker, S., Stoica, I. (2006). Replay debugging for distributed applications. USENIX, p. 289--300.

Digital Library

[9]

Hager, G. D., Toyama, K. (1998). X Vision: A portable substrate for real-time vision applications. Computer Vision and Image Understanding, 69(1), p. 23--37.

Digital Library

[10]

Hartmann, B., Abdulla, L., Mittal, M., Klemmer, S. R. (2007). Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. CHI, p. 145--154.

Digital Library

[11]

Hartmann, B., Klemmer, S. R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., Gee, J. (2006). Reflective physical prototyping through integrated design, test, and analysis. UIST, p. 299--308.

Digital Library

[12]

Hartmann, B., Yu, L., Allison, A., Yang, Y., Klemmer, S. R. (2008). Design as exploration: creating interface alternatives through parallel authoring and runtime tuning. UIST, p. 91--100.

Digital Library

[13]

Klemmer, S. R., Li, J., Lin, J., Landay, J. A. (2004). Papier-Mache: toolkit support for tangible input. CHI, p. 399--406.

Digital Library

[14]

Landay, J. and Myers, B. (1995). Interactive sketching for the early stages of user interface design. CHI, p. 43--50.

Digital Library

[15]

Maynes-Aminzade, D., Pausch, R., Seitz, S. (2002). Techniques for interactive audience participation. ICMI, p. 15--20.

Digital Library

[16]

Maynes-Aminzade, D., Winograd, T., Igarashi, T. (2007). Eyepatch: Prototyping Camera-based Interaction Through Examples. UIST, p. 33--42.

Digital Library

[17]

Max/MSP. Cycling '74. http://cycling74.com/products/maxmsp

[18]

McDirmid, S. (2007). Living it up with a live programming language. OOPSLA, p. 623--638.

Digital Library

[19]

Microsoft Visual Studio Visualizers. http://msdn.microsoft.com/en-us/library/zayyhzts.aspx

[20]

Moher, T. G. (1988). PROVIDE: a process visualization and debugging environment. IEEE Transactions on Software Engineering, 14(6), p. 849--857.

Digital Library

[21]

Newman, M. W., Ackerman, M. S., Kim, J., Prakash, A., Hong, Z., Mandel, J., Dong, T. (2010). Bringing the field into the lab: supporting capture and replay of contextual data for the design of context-aware applications. UIST, p. 105--108.

Digital Library

[22]

Norman, D. A., Draper, S. W. (1986). User centered system design; new perspectives on human-computer interaction. L. Erlbaum Assoc. Inc.

Digital Library

[23]

Patel, K., Bancroft, N., Drucker, S. M., Fogarty, J., Ko, A. J., Landay, J. (2010). Gestalt: integrated support for implementation and analysis in machine learning. UIST, p. 37--46.

Digital Library

[24]

Pothier, G., Tanter, E., Piquer, J. (2007). Scalable omniscient debugging. SIGPLAN, p. 535--552.

Digital Library

[25]

Segen, J., Kumar, S. (1998). Gesture VR: Vision-based 3D hand interface for spatial interaction. Multimedia. p. 455--464.

Digital Library

[26]

SharpDevelop. http://www.icsharpcode.net/opensource/sd/

[27]

Visan, A., Arya, K., Cooperman, G., Denniston, T. (2011). URDB: a universal reversible debugger based on decomposing debugging histories. PLOS, p. 1--5.

Digital Library

[28]

Wang, S., Xiong, X., Xu, Y., Wang, C., Zhang, W., Dai, X., Zhang, D. (2006). Face-tracking as an augmented input in video games: enhancing presence, role-playing and control. CHI, p. 1097--1106.

Digital Library

[29]

Wilson, A. D. (2005). PlayAnywhere: a compact interactive tabletop projection-vision system. UIST, p. 83--92.

Digital Library

[30]

Zeller, A., Lütkehaus, D. (1996). DDD - a free graphical front-end for UNIX debuggers. SIGPLAN Notices, 31(1), p. 22--27.

Digital Library

Cited By

Chugh GChakraborty SChakraborty S(2025)Unlocking Eye Gestures with Earable Inertial Sensing for Accessible HCI2025 17th International Conference on COMmunication Systems and NETworks (COMSNETS)10.1109/COMSNETS63942.2025.10885707(828-832)Online publication date: 6-Jan-2025
https://doi.org/10.1109/COMSNETS63942.2025.10885707
Krebs ERein PBergsiek JUrban LHirschfeld R(2023)Probe Log: Visualizing the Control Flow of Babylonian ProgrammingCompanion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming10.1145/3594671.3594679(61-67)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3594671.3594679
Jiang PSun FXia H(2023)Log-it: Supporting Programming with Interactive, Contextual, Structured, and Visual LogsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581403(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581403
Show More Cited By

Index Terms

DejaVu: integrated support for developing interactive camera-based programs
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

DejaVu: deterministic Java replay debugger for Jalapeño Java virtual machine
OOPSLA '00: Addendum to the 2000 proceedings of the conference on Object-oriented programming, systems, languages, and applications (Addendum)

The execution behavior of a Java application can be non-deterministic due to multithreading. This non-determinism makes understanding and debugging multithreaded Java applications a difficult and laborious process. DejaVu (Deterministic Java Replay ...
Degree-of-knowledge: Modeling a developer's knowledge of code

As a software system evolves, the system's codebase constantly changes, making it difficult for developers to answer such questions as who is knowledgeable about particular parts of the code or who needs to know about changes made. In this article, we ...
DejaVu: visual diffing of cyber physical systems
BuildSys '17: Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments

In this abstract we present DejaVu, a 3D virtual world co-simulator for 'visual diffing' of cyber-physical system deployments in indoor and outdoor environments. Using faster-than-real-time simulation and efficient recording DejaVu can record days of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

October 2012

608 pages

ISBN:9781450315807

DOI:10.1145/2380116

General Chair:
Rob Miller
MIT CSAIL, USA
,
Program Chairs:
Hrvoje Benko
Microsoft Research, USA
,
Celine Latulipe
University of North Carolina at Charlotte, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '12

Sponsor:

UIST '12: The 25th Annual ACM Symposium on User Interface Software and Technology

October 7 - 10, 2012

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
693
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chugh GChakraborty SChakraborty S(2025)Unlocking Eye Gestures with Earable Inertial Sensing for Accessible HCI2025 17th International Conference on COMmunication Systems and NETworks (COMSNETS)10.1109/COMSNETS63942.2025.10885707(828-832)Online publication date: 6-Jan-2025
https://doi.org/10.1109/COMSNETS63942.2025.10885707
Krebs ERein PBergsiek JUrban LHirschfeld R(2023)Probe Log: Visualizing the Control Flow of Babylonian ProgrammingCompanion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming10.1145/3594671.3594679(61-67)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3594671.3594679
Jiang PSun FXia H(2023)Log-it: Supporting Programming with Interactive, Contextual, Structured, and Visual LogsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581403(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581403
Takahashi SIzawa YMasuhara HCong Y(2022)An Approach to Collecting Object Graphs for Data-structure Live Programming Based on a Language Implementation FrameworkJournal of Information Processing10.2197/ipsjjip.30.45130(451-463)Online publication date: 2022
https://doi.org/10.2197/ipsjjip.30.451
Xia HGlueck MAnnett MWang MWigdor D(2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
https://dl.acm.org/doi/10.1145/3503537
Saquib NHuq FHaque S(2022)graphiti: Sketch-based Graph Analytics for Images and VideosProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501923(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501923
Schoop EHuang FHartmann BKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)UMLAUT: Debugging Deep Learning Programs using Program Structure and Model BehaviorProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445538(1-16)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445538
Ikarashi YRagan-Kelley JFukusato TKato JIgarashi T(2021)Guided Optimization for Image Processing Pipelines2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC51201.2021.9576341(1-5)Online publication date: 10-Oct-2021
https://doi.org/10.1109/VL/HCC51201.2021.9576341
Asai KFukusato TIgarashi TBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Integrated Development Environment with Interactive Scatter Plot for Examining Statistical ModelingProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376455(1-7)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376455
Porfirio DFisher ESauppé AAlbarghouthi AMutlu BGuimbretière FBernstein MReinecke K(2019)Bodystorming Human-Robot InteractionsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology10.1145/3332165.3347957(479-491)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3332165.3347957
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten