Skip to main content
SpringerLink
Log in

Wrist-Worn Sensor-Based Tangible Interface for Virtual Percussion Instruments

  • Conference paper
  • First Online:
Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10850))

Abstract

Virtual musical instruments (VMIs) are Augmented Reality (AR) or Virtual Reality (VR) based applications that focus on the interaction with real or virtual musical objects through AR or VR mediums. For such purpose, different interaction tools and techniques such as fiducial markers, RGB and depth cameras, VR and AR headsets are used. Though modern AR and VR devices provide better interaction interfaces, they are more expensive than traditional ones that are unfortunately not robust as the modern devices. To provide a robust and affordable AR and VR interaction interface for real-time applications, we investigate the use of affordable body-worn inertial measurement unit (IMU), a web camera, an RGB light as a simple, cheaper and reliable solution for the interaction with virtual percussion instruments. The performance of the proposed system was evaluated through a series of feedback from fifteen users. In general, the framework was considered as intuitive and tangible.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Poupyrev, I., Berry, R., Kurumisawa, J., Nakao, K., Billinghurst, M., Airola, C., Kato, H., Yonezawa, T., Baldwin, L.: Augmented groove: collaborative jamming in augmented reality. In: ACM SIGGRAPH 2000 Conference Abstracts and Applications, p. 77 (2000)

    Google Scholar 

  2. Poupyrev, I., Berry, R., Billinghurst, M., Kato, H., Nakao, K., Baldwin, L., Kurumisawa, J.: Augmented reality interface for electronic music performance. In: Proceedings of HCI, pp. 805–808 (2001)

    Google Scholar 

  3. Jordà, S.: Interactive music systems for everyone: exploring visual feedback as a way for creating more intuitive, efficient and learnable instruments. In: Proceedings of the Stockholm Music Acoustics Conference, pp. 6–9, August 2003

    Google Scholar 

  4. Karjalainen, M., Maki-Patola, T.: Physics-based modeling of musical instruments for interactive virtual reality. In: 2004 IEEE 6th Workshop on Multimedia Signal Processing, pp. 223–226. IEEE (2004)

    Google Scholar 

  5. Wöldecke, B., Marinos, D., Pogscheba, P., Geiger, C., Herder, J., Schwirten, T.: RadarTHEREMIN-Creating musical expressions in a virtual studio environment. In: 2011 IEEE International Symposium on VR Innovation (ISVRI), pp. 345–346. IEEE (2011)

    Google Scholar 

  6. Olivieri, D., Conde, I.G., Vila-Sobrino, X.A.: AR-based virtual musical instruments using SMC tracking. In: 2011 6th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–7. IEEE (2011)

    Google Scholar 

  7. Rastogi, A., Joshi, A.: Virtual Musical Instruments (2015)

    Google Scholar 

  8. Lobo, N.: V-Drum: an augmented reality drum kit. Int. J. Adv. Res. Comput. Commun. Eng. 4(10), October 2015

    Google Scholar 

  9. Yamabe, T., Asuma, H., Kiyono, S., Nakajima, T.: Feedback design in augmented musical instruments: a case study with an ar drum kit. In: 2011 IEEE 17th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), vol. 2, pp. 126–129. IEEE (2011)

    Google Scholar 

  10. Nilsson, S., Vechev, V., Yeh, A., Hedler, C.: Holo beats: design and development of an AR system to teach drums. In: Proceedings of SIDeR 2016, The 12th Student Interaction Design Research Conference, Malmö (2016)

    Google Scholar 

  11. Fernandez, C.A.T., Paliyawan, P., Yin, C.C., Thawonmas, R.: Piano learning application with feedback provided by an AR virtual character. In: 2016 IEEE 5th Global Conference on Consumer Electronics, pp. 1–2. IEEE (2016)

    Google Scholar 

  12. Cabral, M., Montes, A., Roque, G., Belloc, O., Nagamura, M., Faria, R.R.A., Teubl, F., Kurashima, C., Lopes, R., Zuffo, M.: Crosscale: a 3D virtual musical instrument interface. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 199–200. IEEE (2015)

    Google Scholar 

  13. Fillwalk, J.: Chromachord: a virtual musical instrument. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 201–202. IEEE (2015)

    Google Scholar 

  14. Moore, A.G., Howell, M.J., Stiles, A.W., Herrera, N.S., McMahan, R.P.: Wedge: a musical interface for building and playing composition-appropriate immersive environments. In: 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 205–206. IEEE (2015)

    Google Scholar 

  15. Hariadi, R.R., Kuswardayan, I.: Design and implementation of Virtual Indonesian Musical Instrument (VIMi) application using Leap Motion Controller. In: 2016 International Conference on Information & Communication Technology and Systems (ICTS), pp. 43–48. IEEE (2016)

    Google Scholar 

  16. Hsu, M.-H., Kumara, W.G.C.W., Shih, T.K., Cheng, Z.: Spider King: virtual musical instruments based on microsoft kinect. In: 2013 International Joint Conference on Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA), pp. 707–713. IEEE (2013)

    Google Scholar 

  17. Burks, N., Smith, L., Saquer, J.: A virtual xylophone for music education. In: 2016 IEEE International Symposium on Multimedia (ISM), pp. 409–410. IEEE (2016)

    Google Scholar 

  18. Correa, A.G.D., Ficheman, I.K., do Nascimento, M., de Deus Lopes, R.: Computer assisted music therapy: a case study of an augmented reality musical system for children with cerebral palsy rehabilitation. In: Ninth IEEE International Conference on Advanced Learning Technologies, ICALT 2009, pp. 218–220. IEEE (2009)

    Google Scholar 

  19. Chouvatut, V., Jindaluang, W.: Virtual piano with real-time interaction using automatic marker detection. In: 2013 International Computer Science and Engineering Conference (ICSEC), pp. 222–226. IEEE (2013)

    Google Scholar 

  20. MbientLab. https://mbientlab.com/product/metamotionr/

  21. MbientLab. https://mbientlab.com/cppdocs/latest/dataprocessor.html#pulse

  22. Guitar Center. http://www.guitarcenter.com/Sound-Percussion-Labs/Bell-Kit-w-Rolling-Cart-2-1-2-OCTAVE-1385392679518.gc?pfm=item_page.rrt1|ClickCP

  23. Guitar Center. http://www.guitarcenter.com/Sound-Percussion-Labs/Kicker-Pro-5-Piece-Drum-Set-with-Stands-Cymbals-and-Throne-Silver-Metallic-Glitter-1395067583347.gc?cntry = us&source = 4WWRWXGP&gclid = Cj0KCQiAh_DTBRCTARIsABlT9MYwbMgfMMDaQJZ91WMf6cj6G_4SnvzlOcfDEA2EGzIKW9p2afuIz1UaArMCEALw_wcB&kwid = productads-adid^78244844802-device^c-plaid^146600819322-sku^1395067583347@ADL4GC-adType^PLA

  24. Amazon. https://www.amazon.com/HTC-VIVE-Virtual-Reality-System-pc/dp/B00VF5NT4I/ref=sr_1_1?s=electronics&ie=UTF8&qid=1518159685&sr=1-1&keywords=htc%2Bvive&th=1

  25. Amazon. https://www.amazon.com/Oculus-Rift-Virtual-Reality-Headset-pc/dp/B00VF0IXEY/ref=sr_1_4?s=electronics&ie=UTF8&qid=1518159763&sr=1-4&keywords=oculus+rift

  26. Amazon. https://www.amazon.com/Microsoft-Windows-Development-Requires-Dedicated/dp/B00KZIVEXO/ref=sr_1_1?ie=UTF8&qid=1518420355&sr=8-1&keywords=kinect+windows+v2

  27. Amazon. https://www.amazon.com/MBIENTLAB-Meta-Motion-Sensor-Fusion/dp/B01N9C3HVU/ref=sr_1_1?s=electronics&ie=UTF8&qid=1518420762&sr=1-1&keywords=metamotion+r

  28. Taobao. https://item.taobao.com/item.htm?spm=a1z09.2.0.0.34792e8dG0R4A6&id=558971532837&_u=c8icdh45a36

  29. Tmall. https://detail.tmall.com/item.htm?id=12513799511&spm=a1z09.2.0.0.34792e8dG0R4A6&_u=c8icdh47c30&skuId=3433523876996

Download references

Acknowledgments

The research was sponsored by CAS-TWAS President’s Fellowship.

Author information

Authors and Affiliations

  1. School of Software Engineering, University of Science and Technology of China, Hefei, Anhui, China

    Abassin Sourou Fangbemi

  2. Department of Communication of Science and Technology, University of Science and Technology of China, Hefei, Anhui, China

    Yanxiang Zhang

Authors
  1. Abassin Sourou Fangbemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanxiang Zhang .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Paris-Sud, Orsay, France

    Patrick Bourdot

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fangbemi, A.S., Zhang, Y. (2018). Wrist-Worn Sensor-Based Tangible Interface for Virtual Percussion Instruments. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018. Lecture Notes in Computer Science(), vol 10850. Springer, Cham. https://doi.org/10.1007/978-3-319-95270-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95270-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95269-7

  • Online ISBN: 978-3-319-95270-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation