Richard Savery
ABSTRACT
This paper describes a prototype audio-visual performance of a Ufactory Uarm Swift and a live musician. In this setting, the robotic arm was used as an AI agent to create a visual representation of a musical work in real-time. An A4 white canvas was gradually filled with a mixture of black, blue, red and yellow paints across the span of approximately eight minutes. The musician, performing on an acoustic violin, fitted with a custom built audio interface, performed multiple versions of an improvisatory work developed specifically for the prototype performance. The following sections discuss our technical approach to programming and implementing the Ufactory Uarm Swift as a painting arm, reflections of the musical process and propose future directions for this project.
- Theodor W Adorno and Susan Gillespie. 1995. On some relationships between music and painting. The Musical Quarterly 79, 1 (1995), 66–79.Google Scholar
Cross Ref
- Rong Chang and Yiyuan Huang. 2021. Towards AI Aesthetics: Human-AI Collaboration in Creating Chinese Landscape Painting. In International Conference on Human-Computer Interaction. Springer, 213–224.Google Scholar
- Martin Daniel Cooney and Maria Luiza Recena Menezes. 2018. Design for an art therapy robot: An explorative review of the theoretical foundations for engaging in emotional and creative painting with a robot. Multimodal Technologies and Interaction 2, 3 (2018), 52.Google ScholarCross Ref
- JRV Sai Kiran and S Prabhu. 2020. Robot Nano Spray Painting-A Review. In IOP Conference series: materials science and engineering, Vol. 912. IOP Publishing, 032044.Google Scholar
- Shunsuke Kudoh, Koichi Ogawara, Miti Ruchanurucks, and Katsushi Ikeuchi. 2009. Painting robot with multi-fingered hands and stereo vision. Robotics and Autonomous Systems 57, 3 (2009), 279–288.Google ScholarDigital Library
- Amy Laviers and Cat Maguire. 2022. The BESST System: Explicating a New Component of Time in Laban/Bartenieff Movement Studies Through Work With Robots. In Proceedings of the 8th International Conference on Movement and Computing. 1–3.Google ScholarDigital Library
- Thomas Lindemeier, Jens Metzner, Lena Pollak, and Oliver Deussen. 2015. Hardware-Based Non-Photorealistic Rendering Using a Painting Robot. In Computer graphics forum, Vol. 34. Wiley Online Library, 311–323.Google Scholar
- Luc Nijs. 2018. Dalcroze meets technology: integrating music, movement and visuals with the Music Paint Machine. Music Education Research 20, 2 (2018), 163–183.Google ScholarCross Ref
- Richard Savery, Lisa Zahray, and Gil Weinberg. 2020. Shimon the Rapper: A Real-Time System for Human-Robot Interactive Rap Battles. In International Conference on Computational Creativity, ICCC20.Google Scholar
- Richard Savery, Lisa Zahray, and Gil Weinberg. 2021. Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities. Frontiers in Robotics and AI 8 (2021), 116.Google ScholarCross Ref
- Patrick Tresset and Frederic Fol Leymarie. 2013. Portrait drawing by Paul the robot. Computers & Graphics 37, 5 (2013), 348–363.Google ScholarDigital Library
Index Terms
- Robotic Arm Generative Painting Through Real-time Analysis of Music Performance
Recommendations
Generative Audio and Real-Time Soundtrack Synthesis in Gaming Environments: An exploration of how dynamically rendered soundtracks can introduce new artistic sound design opportunities and enhance the immersion of interactive audio spaces.
OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer InteractionAn important yet oft-overlooked front in the scope of interactive media, audio technologies have remained relatively stagnant compared to groundbreaking advancements made in fields such as visual fidelity and virtual reality. This paper explores the use ...
Resynchronize Japanese "Geisha" Dance Video Using Music of Different Styles
CULTURECOMPUTING '13: Proceedings of the 2013 International Conference on Culture and ComputingMusic and dancing are two different arts yet inseparable and which can be both powerful expression channels for a society or an artist. In most cases, the rhythm, tempo and performance of a dance depend on those of the music. This paper presents a new ...
Melody Transcription From Music Audio: Approaches and Evaluation
Although the process of analyzing an audio recording of a music performance is complex and difficult even for a human listener, there are limited forms of information that may be tractably extracted and yet still enable interesting applications. We ...
Comments