Jingyi Li
Joel Brandt
ABSTRACT
Programming offers new opportunities for visual art creation, but understanding and manipulating the abstract representations that make programming powerful can pose challenges for artists who are accustomed to manual tools and concrete visual interaction. We hypothesize that we can reduce these barriers through programming environments that link state to visual artwork output. We created Demystified Dynamic Brushes (DDB), a tool that bidirectionally links code, numerical data, and artwork across the programming interface and the execution environment - i.e., the artist's in-progress artwork. DDB automatically records stylus input as artists draw, and stores a history of brush state and output in relation to the input. This structure enables artists to inspect current and past numerical input, state, and output and control program execution through the direct selection of visual geometric elements in the drawing canvas. An observational study suggests that artists engage in program inspection when they can visually access geometric state information on the drawing canvas in the process of manual drawing.
Supplemental Material
- Cycling '74. 2016. Max. (2016). http://cycling74.com/products/max.Google Scholar
- Michel Beaudouin-Lafon and Wendy E. Mackay. 2000. Reification, Polymorphism and Reuse: Three Principles for Designing Visual Interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '00). ACM, NY, NY, USA, 102--109. DOI: http://dx.doi.org/10.1145/345513.345267Google ScholarDigital Library
- John Berger. 2008. Drawing. In Selected Essays of John Berger, Geoff Dyer (Ed.). Knopf Doubleday Publishing Group.Google Scholar
- Brian Burg, Richard Bailey, Amy J. Ko, and Michael D. Ernst. 2013. Interactive Record/Replay for Web Application Debugging. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 473--484. DOI: http://dx.doi.org/10.1145/2501988.2502050Google ScholarDigital Library
- Ellen Do and Mark D. Gross. 2007. Environments for Creativity: A Lab for Making Things. In Proceedings of the 6th ACM SIGCHI Conference on Creativity & Cognition (C&C '07). ACM.Google Scholar
- Max Documentation. 2019. Max Signal Probing. (2019). https://docs.cycling74.com/max7/vignettes/signal_probe.Google Scholar
- Benjamin Jotham Fry. 2004. Computational information design. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- David J. Gilmore. 1991. Models of debugging. Acta Psychologica 78, 1 (1991), 151--172. DOI: http://dx.doi.org/https://doi.org/10.1016/0001--6918(91)90009-OGoogle ScholarCross Ref
- M. D. Gross. 2009. Visual Languages and Visual Thinking: Sketch Based Interaction and Modeling. In Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling (SBIM '09). ACM, NY, NY, USA, 7--11. DOI: http://dx.doi.org/10.1145/1572741.1572743Google ScholarDigital Library
- Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R. Klemmer. 2008. Design As Exploration: Creating Interface Alternatives Through Parallel Authoring and Runtime Tuning. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, NY, NY, USA, 91--100. http://doi.acm.org/10.1145/1449715.1449732Google Scholar
- B. Harvey. 1991. Symbolic Programming vs. the A.P. Curriculum. The Computing Teacher 56 (February 1991), 27--29.Google Scholar
- Brian Hempel and Ravi Chugh. 2016. Semi-Automated SVG Programming via Direct Manipulation. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 379--390. DOI: http://dx.doi.org/10.1145/2984511.2984575Google ScholarDigital Library
- Joshua Hibschman and Haoqi Zhang. 2016. Telescope: Fine-Tuned Discovery of Interactive Web UI Feature Implementation. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 233--245. DOI: http://dx.doi.org/10.1145/2984511.2984570Google ScholarDigital Library
- Jane Hoffswell, Arvind Satyanarayan, and Jeffrey Heer. 2018. Augmenting Code with In Situ Visualizations to Aid Program Understanding. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 532, 12 pages. DOI: http://dx.doi.org/10.1145/3173574.3174106Google ScholarDigital Library
- Tim Ingold. 2010. The textility of making. Cambridge Journal of Economics 34 (01 2010), 91--102. DOI: http://dx.doi.org/10.1093/cje/bep042Google ScholarCross Ref
- Jennifer Jacobs, Joel Brandt, Radomír Mech, and Mitchel Resnick. 2018. Extending Manual Drawing Practices with Artist-Centric Programming Tools. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 590, 13 pages. DOI: http://dx.doi.org/10.1145/3173574.3174164Google ScholarDigital Library
- Jennifer Jacobs, Sumit Gogia, Radomír M?ch, and Joel R. Brandt. 2017. Supporting Expressive Procedural Art Creation Through Direct Manipulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 6330--6341. DOI: http://dx.doi.org/10.1145/3025453.3025927Google ScholarDigital Library
- Scott R. Klemmer, Björn Hartmann, and Leila Takayama. 2006. How Bodies Matter: Five Themes for Interaction Design. In Proceedings of the 6th Conference on Designing Interactive Systems (DIS '06). ACM, NY, NY, USA, 140--149. DOI: http://dx.doi.org/10.1145/1142405.1142429Google ScholarDigital Library
- Amy J. Ko and Brad A. Myers. 2008. Debugging Reinvented: Asking and Answering Why and Why Not Questions About Program Behavior. In Proceedings of the 30th International Conference on Software Engineering (ICSE '08). ACM, NY, NY, USA, 301--310. DOI: http://dx.doi.org/10.1145/1368088.1368130Google ScholarDigital Library
- Mode Lab. 2015. What is a Data Tree? In The Grasshopper Primer (3rd ed.). https://modelab.gitbooks.io/grasshopper-primer/ 1-foundations/1--5/2_what-is-a-data-tree.html.Google Scholar
- Tom Lieber, Joel R. Brandt, and Rob C. Miller. 2014. Addressing Misconceptions About Code with Always-on Programming Visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2481--2490. DOI: http://dx.doi.org/10.1145/2556288.2557409Google ScholarDigital Library
- Henry Lieberman. 1984. Steps Toward Better Debugging Tools for LISP. In Proceedings of the 1984 ACM Symposium on LISP and Functional Programming (LFP '84). ACM, NY, NY, USA, 247--255. DOI: http://dx.doi.org/10.1145/800055.802041Google ScholarDigital Library
- Z. Lieberman, T. Watson, and A. Castro. 2015. openFrameworks. (2015). http://openframeworks.cc/about.Google Scholar
- Zhicheng Liu, John Thompson, Alan Wilson, Mira Dontcheva, James Delorey, Sam Grigg, Bernand Kerr, and John Stasko. 2018. Data Illustrator: Augmenting Vector Design Tools with Lazy Data Binding for Expressive Visualization Authoring. In To Appear. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA. DOI: http://dx.doi.org/10.1145/3173574.3173797Google ScholarDigital Library
- Rune Madsen. 2019. Introduction. In Programming Design Systems. https://programmingdesignsystems.com/introduction.Google Scholar
- Michael Mateas. 2005. Procedural literacy: Educating the new media practitioner. On the Horizon 13 (06 2005), 101--111. DOI: http://dx.doi.org/10.1108/10748120510608133Google ScholarCross Ref
- Lauren McCarthy. 2016. P5.js. (2016). http://p5js.org/.Google Scholar
- M. McCullough. 1996. Abstracting Craft: The Practiced Digital Hand. The MIT Press, Cambridge, Massachusetts.Google ScholarDigital Library
- Matt Pearson. 2011. Generative Art. Manning Publications Co., Greenwich, CT, USA.Google Scholar
- Casey Reas. 2004. The Language of Computers. In Creative Code, John Maeda and Red Burns (Eds.). Thames & Hudson, London, United Kingdom, 44.Google Scholar
- C. Reas and B. Fry. 2004. Processing. (2004). http://processing.org.Google Scholar
- C. Reas and B. Fry. 2007. The Processing Handbook. MIT Press, Cambridge, Massachusetts, USA.Google Scholar
- C. Reas and B. Fry. 2019. Processing Environment. (2019). https://processing.org/reference/environment/.Google Scholar
- C. Reas, C. McWilliams, and LUST. 2010. Form and Code. Princeton Architectural Press, New York, NY, USA.Google Scholar
- M. Resnick and E.O. Rosenbaum. 2013. Designing for Tinkerability. In Design Make Play: Growing the Next Generation of STEM Innovators, M. Honey and D. Kanter (Eds.). Routledge.Google Scholar
- David Rutten. 2007. Grasshopper. http://www.grasshopper3d.com. (2007).Google Scholar
- Toby Schachman. 2012. Alternative Programming Interfaces for Alternative Programmers. In Proceedings of the ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2012). ACM, NY, NY, USA, 1--10. DOI: http://dx.doi.org/10.1145/2384592.2384594Google ScholarDigital Library
- Bret Victor. 2011. Dynamic Pictures. (2011). http://worrydream.com/DynamicPicturesMotivation.Google Scholar
- B. Victor. 2012. Learnable Programing: Designing a programming system for understanding programs. (2012). http://worrydream.com/LearnableProgramming/Google Scholar
- vvvv group. 2017. vvvv. (2017). https://vvvv.org/.Google Scholar
- Marius Watz. 2012. The Algorithm Thought Police. http://mariuswatz.com/mwatztumblrcom/ the-algorithm-thought-police.html/. (2012).Google Scholar
- Haijun Xia, Nathalie Henry Riche, Fanny Chevalier, De Araujo, and Daniel Widgor. 2018. DataInk: Direct and Creative Data-Oriented Drawing. In To Appear. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA. DOI: http://dx.doi.org/10.1145/3173574.3173797Google ScholarDigital Library
- Loutfouz Zaman, Wolfgang Stuerzlinger, Christian Neugebauer, Rob Woodbury, Maher Elkhaldi, Naghmi Shireen, and Michael Terry. 2015. GEM-NI: A System for Creating and Managing Alternatives In Generative Design. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 1201--1210. DOI: http://dx.doi.org/10.1145/2702123.2702398Google ScholarDigital Library
Index Terms
- Supporting Visual Artists in Programming through Direct Inspection and Control of Program Execution
Recommendations
What We Can Learn From Visual Artists About Software Development
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsThis paper explores software’s role in visual art production by examining how artists use and develop software. We conducted interviews with professional artists who were collaborating with software developers, learning software development, and building ...
Supporting Expressive Procedural Art Creation through Direct Manipulation
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsComputation is a powerful artistic medium. Artists with experience in programming have demonstrated the unique creative opportunities of using code to make art. Currently, manual artists interested in using procedural techniques must undergo the ...
Visual Art Design for Multimedia Technology Art Work “The Afterlife”
ICEMT '23: Proceedings of the 7th International Conference on Education and Multimedia TechnologyThis creative art work is classified in the field of visual arts. The objective of conveying creativity to reflect religious beliefs, Buddhist philosophy which is a universal truth that exists in all forms, namely impermanence, namely death. From study ...
Comments