skip to main content
research-article

Painting an Apple with an Apple: A Tangible Tabletop Interface for Painting with Physical Objects

Published: 27 December 2018 Publication History

Abstract

We introduce UnicrePaint, a digital painting system that allows the user to paint with physical objects by acquiring three parameters from the interacting object: the form, the color pattern and the contact pressure. The design of the system is motivated by a hypothesis that integrating direct input from physical objects with digital painting offers unique creative experiences to the user. A major technical challenge in implementing UnicrePaint is to resolve the conflict between input and output, i.e., to be able to capture the form and color pattern of contacting objects from a camera, while at the same time be able to present the captured data using a projector. We present a solution for this problem. We implemented a prototype and carried out a user study with fifteen novice users. Additionally, five professional users with art-related backgrounds participated in a user study to obtain insights into how professionals might view our system. The results show that UnicrePaint offers unique experiences with painting in a creative manner. Also, its potentials beyond mere artwork are suggested.

Supplementary Material

fujinami (fujinami.zip)
Supplemental movie, appendix, image and software files for, Painting an Apple with an Apple: A Tangible Tabletop Interface for Painting with Physical Objects

References

[1]
Bill Baxter, Vincent Scheib, Ming C. Lin, and Dinesh Manocha. 2001. DAB: interactive haptic painting with 3D virtual brushes. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '01. ACM Press, New York, New York, USA, 461--468.
[2]
Anthony M. Blatner, James A. Ferwerda, Benjamin A. Darling, and Reynold J. Bailey. 2011. TangiPaint: A Tangible Digital Painting System. In 19th Color and Imaging Conference Final Program and Proceedings. Society for Imaging Science and Technology, 102--107. http://www.ingentaconnect.com/contentone/ist/cic/2011/00002011/00000001/art00022
[3]
Erin Cherry and Celine Latulipe. 2014. Quantifying the Creativity Support of Digital Tools through the Creativity Support Index. ACM Transactions on Computer-Human Interaction 21, 4 (6 2014), 1--25.
[4]
Nelson S.-H. Chu and Chiew-Lan Tai. 2005. MoXi: real-time ink dispersion in absorbent paper. ACM Transactions on Graphics (TOG) 24, 3 (2005), 504--511.
[5]
Nicholas Davis, Holger Winnemöller, Mira Dontcheva, and Ellen Yi-Luen Do. 2013. Toward a Cognitive Theory of Creativity Support. In Proceedings of the 9th ACM Conference on Creativity & Cognition (C&C '13). ACM, New York, NY, USA, 13--22.
[6]
Raoul De Charette, Robert Tamburo, Peter C Barnum, Anthony Rowe, Takeo Kanade, and Srinivasa G Narasimhan. 2012. Fast reactive control for illumination through rain and snow. In Computational Photography (ICCP), 2012 IEEE International Conference on. IEEE, 1--10.
[7]
Florian Echtler, Andreas Dippon, Marcus Tönnis, and Gudrun Klinker. 2009. Inverted FTIR: easy multi-touch sensing for flatscreens. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '09. ACM Press, New York, New York, USA, 29--32.
[8]
embarth. 2017. Eco-printing with rust and vinegar. https://www.instructables.com/id/Eco-Printing-With-Rust-Vinegar/
[9]
Exseal Corporation. 2016. Urethane Gel Series Products Catalog., 28--29 pages. http://www.exseal.co.jp/pdf/catalog_en.pdf
[10]
Sean Follmer and Hiroshi Ishii. 2012. KidCAD: digitally remixing toys through tangible tools. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI '12. ACM Press, New York, New York, USA, 2401--2410.
[11]
Sean Follmer, Micah Johnson, Edward Adelson, and Hiroshi Ishii. 2011. deForm: An Interactive Malleable Surface for Capturing 2.5D Arbitrary Objects, Tools and Touch. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 527--536.
[12]
Jennifer Forker. 2013. Play with your food and stamp it. https://www.statesman.com/lifestyles/play-with-your-food-and-stamp/GCfr6UM5uYgALPjGbmZ0IJ/
[13]
Markus Gross, Silke Lang, Kai Strehlke, Andrew Vande Moere, Oliver Staadt, Stephan Würmlin, Martin Naef, Edouard Lamboray, Christian Spagno, Andreas Kunz, Esther Koller-Meier, Tomas Svoboda, Luc Van Gool, Markus Gross, Stephan Würmlin, Martin Naef, Edouard Lamboray, Christian Spagno, Andreas Kunz, Esther Koller-Meier, Tomas Svoboda, Luc Van Gool, Silke Lang, Kai Strehlke, Andrew Vande Moere, and Oliver Staadt. 2003. Blue-c: a spatially immersive display and 3D video portal for telepresence. In ACM SIGGRAPH 2003 Papers on - SIGGRAPH '03, Vol. 22. ACM Press, New York, New York, USA, 819.
[14]
Jefferson Y Han. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology. ACM Press, Seattle, WA, USA, 115--118.
[15]
Fabian Hennecke, Franz Berwein, and Andreas Butz. 2011. Optical pressure sensing for tangible user interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '11. ACM Press, New York, New York, USA, 45--48.
[16]
Anuruddha Hettiarachchi, Suranga Nanayakkara, Kian Peen Yeo, Roy Shilkrot, and Pattie Maes. 2013. FingerDraw: more than a digital paintbrush. In Proceedings of the 4th Augmented Human International Conference on - AH '13. ACM Press, New York, New York, USA, 1--4.
[17]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '97. ACM Press, New York, New York, USA, 234--241.
[18]
Shahram Izadi, Steve Hodges, Alex Butler, Darren West, Alban Rrustemi, Mike Molloy, and William Buxton. 2009. ThinSight: A Thin Form-Factor Interactive Surface Technology. Commun. ACM 52, 12 (12 2009), 90--98.
[19]
Shahram Izadi, Steve Hodges, Stuart Taylor, Dan Rosenfeld, Nicolas Villar, Alex Butler, and Jonathan Westhues. 2008. Going Beyond the Display: A Surface Technology with an Electronically Switchable Diffuser. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, New York, NY, USA, 269--278.
[20]
Daniel Jackson, Tom Bartindale, and Patrick Olivier. 2009. FiberBoard: compact multi-touch display using channeled light. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '09. ACM Press, New York, New York, USA, 25--28.
[21]
Rubaiat Habib Kazi, Kien Chuan Chua, Shengdong Zhao, Richard Davis, and Kok-Lim Low. 2011. SandCanvas: a multi-touch art medium inspired by sand animation. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems - CHI '11. ACM Press, New York, New York, USA, 1283--1292.
[22]
Mami Kosaka and Kaori Fujinami. 2016. UnicrePaint: Digital Painting through Physical Objects for Unique Creative Experiences. In Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '16. ACM Press, New York, New York, USA, 475--481.
[23]
Xin Liu, Haijun Xia, and Jiawei Gu. 2013. FlexStroke: a jamming brush tip simulating multiple painting tools on digital platform. In Proceedings of the Adjunct Publication of the 26th Annual ACM Symposium on User Interface Software and Technology - UIST '13 Adjunct. ACM Press, New York, New York, USA, 23--24.
[24]
Nobuyuki Matsushita and Jun Rekimoto. 1997. HoloWall: designing a finger, hand, body, and object sensitive wall. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology - UIST '97. ACM Press, Banff, Alberta, Canada, 209--210.
[25]
Ilya D. Rosenberg, Alexander Grau, Charles Hendee, Nadim Awad, and Ken Perlin. 2009. IMPAD: an inexpensive multi-touchpressure acquisition device. In Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '09. ACM Press, New York, New York, USA, 3217--3222.
[26]
Kimiko Ryokai, Michael Jongseon Lee, and Jonathan Micah Breitbart. 2009. Children's storytelling and programming with robotic characters. In Proceeding of the Seventh ACM Conference on Creativity and Cognition - C&C '09. ACM Press, New York, New York, USA, 19--28.
[27]
Kimiko Ryokai and Stefan Marti. 2005. I/O Brush. https://youtu.be/04v_v1gnyO8
[28]
Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2004. I/O brush: drawing with everyday objects as ink. In Proceedings of the 2004 Conference on Human Factors in Computing Systems - CHI '04. ACM Press, New York, New York, USA, 303--310.
[29]
Albrecht Schmidt, Martin Strohbach, Kristof Van Laerhoven, Adrian Friday, and Hans-W Gellersen. 2002. Context Acquisition Based on Load Sensing. In Proceedings of the 6th International Conference on Ubiquitous Computing, HUC'99. Springer, Berlin, Heidelberg, 333--350.
[30]
Roy Shilkrot, Pattie Maes, Joseph A. Paradiso, and Amit Zoran. 2015. Augmented Airbrush for Computer Aided Painting (CAP). ACM Transactions on Graphics 34, 2 (3 2015), 1--11.
[31]
Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (2007), 20--32.
[32]
J. David Smith, T.C. Nicholas Graham, David Holman, and Jan Borchers. 2007. Low-Cost Malleable Surfaces with Multi-Touch Pressure Sensitivity. In Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07). IEEE, 205--208.
[33]
Kenji Sugihara, Mai Otsuki, Asako Kimura, Fumihisa Shibata, and Hideyuki Tamura. 2011. MAI painting brush++: augmenting the feeling of painting with new visual and tactile feedback mechanisms. In Proceedings of the 24th Annual ACM Symposium Adjunct on User Interface Software and Technology - UIST '11 Adjunct. ACM Press, New York, New York, USA, 13--14.
[34]
Masahiro Ura, Masashi Yamada, Mamoru Endo, Shinya Miyazaki, and Takami Yasuda. 2010. A Framework of FTIR Table Pressure Sensing for Simulation of Art Performances. In Proceedings of NICOGRAPH International. 118--123.
[35]
Peter Vandoren, Luc Claesen, Tom Van Laerhoven, Johannes Taelman, Chris Raymaekers, Eddy Flerackers, and Frank Van Reeth. 2009. FluidPaint: an interactive digital painting system using real wet brushes. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '09. ACM Press, New York, New York, USA, 53--56.
[36]
Peter Vandoren, Tom Van Laerhoven, Luc Claesen, Johannes Taelman, Chris Raymaekers, and Frank Van Reeth. 2008. IntuPaint: Bridging the gap between physical and digital painting. In 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems. IEEE, 65--72.

Cited By

View all
  • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
  • (2024)“Tele” Me More: Using Telepresence Charades to Connect Strangers and Exhibits in Different MuseumsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650834(1-8)Online publication date: 11-May-2024
  • (2024)Capturing Contact Surfaces by a Frustrated Total Internal Reflection System Using a Curved Plate for Comparison of the Beginning of Touching Motions by Humanitude Experts and Novices2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10831793(679-684)Online publication date: 6-Oct-2024
  • Show More Cited By

Index Terms

  1. Painting an Apple with an Apple: A Tangible Tabletop Interface for Painting with Physical Objects

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 4
      December 2018
      1169 pages
      EISSN:2474-9567
      DOI:10.1145/3301777
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 December 2018
      Accepted: 01 October 2018
      Revised: 01 August 2018
      Received: 01 February 2018
      Published in IMWUT Volume 2, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Creativity support system
      2. Digital painting system
      3. Frustrated Total Internal Reflection (FTIR)
      4. Tabletop interface
      5. Tangible User Interfaces (TUI)

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      • The HAYAO NAKAYAMA Foundation for Science & Technology and Culture

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)35
      • Downloads (Last 6 weeks)3
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
      • (2024)“Tele” Me More: Using Telepresence Charades to Connect Strangers and Exhibits in Different MuseumsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650834(1-8)Online publication date: 11-May-2024
      • (2024)Capturing Contact Surfaces by a Frustrated Total Internal Reflection System Using a Curved Plate for Comparison of the Beginning of Touching Motions by Humanitude Experts and Novices2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10831793(679-684)Online publication date: 6-Oct-2024
      • (2023)Capturing Contact Surfaces by a Frustrated Total Internal Reflection System using a Curved Plate for Analysis of the Beginning of Humanitude's Touching Motions曲面板を用いた赤外線漏れ全反射方式でのユマニチュードの触れ始めの解析のための接触面情報の取得Journal of the Robotics Society of Japan10.7210/jrsj.41.89341:10(893-896)Online publication date: 2023
      • (2023)How HCI concepts are used in articles featuring interactive digital arts: a literature review.Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638116(1-12)Online publication date: 16-Oct-2023
      • (2023)Immersive Sampling: Exploring Sampling for Future Creative Practices in Media-Rich, Immersive SpacesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596131(212-229)Online publication date: 10-Jul-2023
      • (2022)Colorbo: Envisioned Mandala Coloringthrough Human-AI CollaborationProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511135(15-26)Online publication date: 22-Mar-2022
      • (2021)Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future ChallengesACM Computing Surveys10.1145/346796354:8(1-38)Online publication date: 4-Oct-2021
      • (2021)The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support ToolsProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462050(1817-1833)Online publication date: 28-Jun-2021
      • (2021)Dothraki: Tracking Tangibles Atop Tabletops Through De-Bruijn ToriProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440656(1-10)Online publication date: 14-Feb-2021
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media