Skip to main content
SpringerLink
Log in

Virtual pottery: a virtual 3D audiovisual interface using natural hand motions

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present our approach towards designing and implementing a virtual 3D sound sculpting interface that creates audiovisual results using hand motions in real time. In the interface “Virtual Pottery,” we use the metaphor of pottery creation in order to adopt the natural hand motions to 3D spatial sculpting. Users can create their own pottery pieces by changing the position of their hands in real time, and also generate 3D sound sculptures based on pre-existing rules of music composition. The interface of Virtual Pottery can be categorized by shape design and camera sensing type. This paper describes how we developed the two versions of Virtual Pottery and implemented the technical aspects of the interfaces. Additionally, we investigate the ways of translating hand motions into musical sound. The accuracy of the detection of hand motions is crucial for translating natural hand motions into virtual reality. According to the results of preliminary evaluations, the accuracy of both motion-capture tracking system and portable depth sensing camera is as high as the actual data. We carried out user studies, which took into account information about the two exhibitions along with the various ages of users. Overall, Virtual Pottery serves as a bridge between the virtual environment and traditional art practices, with the consequence that it can lead to the cultivation of the deep potential of virtual musical instruments and future art education programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Beyer and Meier M (2011) “Music interfaces for novice users: composing music on a public display with hand gestures,” in Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2011)

  2. Blake A, Isard M (1994) “3D position, attitude and shape input using video tracking of hands and lips,” in Proceedings of the 21st annual conference on computer graphics and interactive techniques (SIGGRAPH '94). ACM Press, New York, pp 185–192

    Google Scholar 

  3. BlobScanner. Retrieved from http://code.google.com/p/BlobScanner/

  4. Cani MP, Alexis Angelidis A (2006) “Towards virtual clay.” in proceedings of ACM SIGGRAPH 2006 courses. USA, New York, pp 67–83

    Book  Google Scholar 

  5. Da Fontoura Costa L, et al. (2000) Shape analysis and classification: theory and practice. CRC Press

  6. Han Y, Han B (2012) “Virtual Pottery: an interactive audio-visual installation,” in Proceedings of 12th international conferences on New Interfaces on Musical Expression (NIME 2012)

  7. Holz C, Wilson A (2011) “Data miming: Inferring spatial object descriptions from human gesture,” in Proceedings of the 2011 annual conference on Human factors in computing systems (ACM CHI ’11), pp. 811–820

  8. Ip HHS, Kwong B, Law KCK (2005) “BodyMusic: A novel framework design for body-driven music composition,” in Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology. ACM Press, New York, pp 342–345

    Google Scholar 

  9. Ip HHS, Kwong B, Law KCK (2005) “Cyber composer: hand gesture-driven intelligent music composition and generation,” in Proceedings of 11th international multimedia modeling conference, pp.46–52

  10. Kameyama K (1997) “Virtual clay modeling system,” in Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp 197–200

  11. Knopf GK, Igwe PC (2005) Deformable mesh for virtual shape sculpting. Robotics and Computer-Integrated Manufacturing 21:302–311

    Article  Google Scholar 

  12. Korida K, Nishino H, Utsumiya K (1997) “An interactive 3D interface for a virtual ceramic art work environment,” in Proceedings of International Conference on Virtual Systems and Multimedia (VSMM '97), pp. 227–234. IEEE Computer Society

  13. L’Artisan Electronique. Retrieved from http://www.creativeapplications.net/openframeworks/lartisan-electronique-openframeworks/.

  14. Lee J, Han G, Choi S (2008) Haptic Pottery modeling using circular sector element method. Springer Lecture Notes in Computer Science (LNCS) 5024:668–674

    Article  Google Scholar 

  15. Levin G, Lieberman Z (2005) “Sounds from shapes: audiovisual performance with hand silhouette contours in the manual input sessions,” in Proceedings of the 2005 international conference on New Interfaces for Musical Expression (NIME 2005), pp. 115–120

  16. McDonnell KT, Qin H, Wlodarczyk RA (2001) “Virtual clay: a real-time sculpting system with haptic toolkits,” in Proceedings of the 2001 symposium of Interactive 3D graphics, pp.179–190, New York, USA

  17. Mulder and Fels S (1998) “Sound sculpting: manipulating sound through virtual sculpting,” in Proceedings of Western Computer Graphics Symposium

  18. Processing. Retrieved from http://processing.org/

  19. Rigid Body toolkit. Retrieved from http://www.naturalpoint.com/OptiTrack/products/rigidbody-toolkit/

  20. UDP library. Retrieved from http://ubaa.net/shared/processing/udp/

  21. Vinayak, Murugappan S, Piya C, Ramani K (2012) “Handy-Potter: Rapid 3D Shape Exploration through Natural Hand Motions,” in Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2012), August 12–15

  22. Young D (2002) “The Hyperbow controller: Real-time dynamics measurement of violin performance,” in Proceedings of the 2002 conference on New Instruments for Musical Expression (NIME 2002)

Download references

Acknowledgement

These works have been supported by TransLab, University of California, Santa Barbara (UCSB), under the guidance of Prof. Marcos Novak.

Author information

Authors and Affiliations

  1. Media Arts and Technology, University of California, Santa Barbara, Santa Barbara, USA

    Yoon Chung Han

  2. School of Electrical Engineering, Korea University, Seoul, South Korea

    Byeong-jun Han

Authors
  1. Yoon Chung Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Byeong-jun Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoon Chung Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, Y.C., Han, Bj. Virtual pottery: a virtual 3D audiovisual interface using natural hand motions. Multimed Tools Appl 73, 917–933 (2014). https://doi.org/10.1007/s11042-013-1382-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1382-3

Keywords

Search

Navigation