skip to main content
10.1145/1753326.1753401acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

FrameWire: a tool for automatically extracting interaction logic from paper prototyping tests

Published: 10 April 2010 Publication History

Abstract

Paper prototyping offers unique affordances for interface design. However, due to its spontaneous nature and the limitations of paper, it is difficult to distill and communicate a paper prototype design and its user test findings to a wide audience. To address these issues, we created FrameWire, a computer vision-based system that automatically extracts interaction flows from the video recording of paper prototype user tests. Based on the extracted logic, FrameWire offers two distinct benefits for designers: a structural view of the video recording that allows a designer or a stakeholder to easily distill and understand the design concept and user interaction behaviors, and automatic generation of interactive HTML-based prototypes that can be easily tested with a larger group of users as well as "walked through" by other stakeholders. The extraction is achieved by automatically aggregating video frame sequences into an interaction flow graph based on frame similarities and a designer-guided clustering process. The results of evaluating FrameWire with realistic paper prototyping tests show that our extraction approach is feasible and FrameWire is a promising tool for enhancing existing prototyping practice.

Supplementary Material

JPG File (p503.jpg)
MOV File (p503.mov)

References

[1]
Alvarado, C. and Davis, R. SketchREAD: a multi-domain sketch recognition engine. UIST'04. p. 23--32.
[2]
Anoto. http://www.anoto.com.
[3]
Bailey, B.P., et al. DEMAIS: designing multimedia applications with interactive storyboards. ACM Multimedia'01. p. 241--250.
[4]
Barnes, C., et al. Video puppetry: a performative interface for cutout animation. ACM Trans. Graph. 27, 5 (Dec. 2008), 1--9.
[5]
Cypher, A., ed. Watch What I Do: Programming by Demonstration. 1993, MIT Press.
[6]
Guimbretière, F. Paper Augmented Digital Documents. UIST'03. p. 51--60.
[7]
Hartmann, B., et al. Reflective physical prototyping through integrated design, test, and analysis. UIST'06. p. 299--308.
[8]
Hong, J.I. and Landay, J.A. WebQuilt: a framework for capturing and visualizing the web experience. WWW'01. p. 717--724.
[9]
Johnson, W., et al. Bridging the paper and electronic worlds: the paper user interface. CHI'93. p. 507--512.
[10]
Kim, J., et al. Video-based document tracking: unifying your physical and electronic desktops. UIST'04. p. 99--107.
[11]
Klemmer, S.R., et al. Integrating Physical and Digital Interactions on Walls for Fluid Design Collaboration. Human-Computer Interaction, 23: p. 138--213.
[12]
Landay, J.A. and Myers, B.A. Sketching Interfaces: Toward More Human Interface Design. IEEE Computer, 2001. 34(3): p. 56--64.
[13]
Liao, C., et al. Papiercraft: A gesture-based command system for interactive paper. TOCHI, 2008. 14(4): p. 1073--0516.
[14]
Lieberman, H., ed. Your Wish Is My Command: Programming by Example. 2001, Morgan Kaufmann.
[15]
MacIntyre, B., et al. DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences. UIST'04. p. 197--206.
[16]
Mackay, W.E. and Fayard, A.L. Video brainstorming and prototyping: techniques for participatory design. In CHI '99 Extended Abstracts. CHI '99. p. 118--119.
[17]
Newman, M.W., et al. DENIM: An Informal Web Site Design Tool Inspired by Observations of Practice. Human-Computer Interaction, 2003. 18(3): p. 259--324.
[18]
Rettig, M. Prototyping for Tiny Fingers. Communications of the ACM, 1994. 37(4): p. 21--27.
[19]
Snyder, C. Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. 2003, Morgan Kaufmann.
[20]
Ward, J.H. Hierarchical Grouping to Optimize an Objective Function. J. Am. Statist. Assoc., 1963. 58: p. 236--244.
[21]
Wellner, P. Interacting with paper on the DigitalDesk. Communications of the ACM, 1993. 36(7): p. 87--96.
[22]
Yeh, R., et al. ButterflyNet: a mobile capture and access system for field biology research. CHI'06. p. 571--580.

Cited By

View all
  • (2022)A Survey on the Use of Computer Vision to Improve Software Engineering TasksIEEE Transactions on Software Engineering10.1109/TSE.2020.303298648:5(1722-1742)Online publication date: 1-May-2022
  • (2021)CoCapture: Effectively Communicating UI Behaviors on Existing Websites by Demonstrating and RemixingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445573(1-14)Online publication date: 6-May-2021
  • (2019)StateLensProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology - UIST '1910.1145/3332165.3347873(371-385)Online publication date: 2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
April 2010
2690 pages
ISBN:9781605589299
DOI:10.1145/1753326
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. paper prototyping
  2. programming by demonstration

Qualifiers

  • Research-article

Conference

CHI '10
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)A Survey on the Use of Computer Vision to Improve Software Engineering TasksIEEE Transactions on Software Engineering10.1109/TSE.2020.303298648:5(1722-1742)Online publication date: 1-May-2022
  • (2021)CoCapture: Effectively Communicating UI Behaviors on Existing Websites by Demonstrating and RemixingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445573(1-14)Online publication date: 6-May-2021
  • (2019)StateLensProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology - UIST '1910.1145/3332165.3347873(371-385)Online publication date: 2019
  • (2019)EnactACM Transactions on Computer-Human Interaction10.1145/331027626:3(1-48)Online publication date: 31-May-2019
  • (2018)SketchStudioProceedings of the 2018 Designing Interactive Systems Conference10.1145/3196709.3196736(831-843)Online publication date: 8-Jun-2018
  • (2018)Storyboard-Based Empirical Modeling of Touch Interface PerformanceProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3174019(1-12)Online publication date: 21-Apr-2018
  • (2018)Extracting Paper Sticky Notes with Visual-Inertial Odometry of ARKit2018 5th International Conference on Computational Science/ Intelligence and Applied Informatics (CSII)10.1109/CSII.2018.00014(37-42)Online publication date: Jul-2018
  • (2016)Gesture morpherProceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/2935334.2935391(227-232)Online publication date: 6-Sep-2016
  • (2016)Enhancing Cross-Device Interaction Scripting with Interactive IllustrationsProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858382(5482-5493)Online publication date: 7-May-2016
  • (2016)Paper Prototyping for BatiKids: A Technique to Examine Children’s Interaction and Feedback in Designing a Game-Based LearningProceedings of Second International Conference on Electrical Systems, Technology and Information 2015 (ICESTI 2015)10.1007/978-981-287-988-2_49(447-455)Online publication date: 30-Jan-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media