skip to main content
10.1145/3173225.3173235acmconferencesArticle/Chapter ViewAbstractPublication PagesteiConference Proceedingsconference-collections
research-article

Real-time Recognition of Guitar Performance Using Two Sensor Groups for Interactive Lesson

Published: 18 March 2018 Publication History

Abstract

The accurate recognition of guitarist performance is challenging compared with other instruments because a guitar player typically plays several notes at once and uses both hands in different ways. In this paper, we propose a sensor-based guitar that consists of two groups of sensors. One sensor is used to recognize the fingering positions of the fretting hand, and the other is used to detect the guitar strings that are played by the picking hand. We design an embedded system for accurate sensing and propose a data analysis mechanism to precisely figure out the played pitch and the duration of notes using the sensed data. We realize our scheme as a high-quality prototype that detects guitarist performance with accuracy sufficient for the transcribing of a performance. We also present real application examples such as a rhythm game for interactive lessons and a music sharing feature with user created musical scores.

References

[1]
Juan Pablo Bello, Laurent Daudet, Samer Abdallah, Chris Duxbury, Mike Davies, and Mark B. Sandler. 2005. A Tutorial on Onset Detection in Musical Signals. IEEE Transactions on Speech and Audio Processing. 13, 5: 1035--1047.
[2]
Frédéric Bevilacqua, Nicolas Rasamimanana, Emmanuel Fléty, Serge Lemounton, and Florence Baschet. 2006. The augmented violin project: research, composition and performance report. In Proceedings of the international conference on New Interfaces for Musical Expression (NIME '06), 402--406. http://nime.org/proceedings/2006/nime2006_402.pdf
[3]
Giovanni Costantini, Massimiliano Todisco, and Giovanni Saggio. 2011. A sensor interface based on sparse NMF for piano musical transcription. In Advances in Sensors and Interfaces (IWASI), 157--161.
[4]
Fretlight. 2017. Wireless Guitar. (Dec 2017). http://fretlight.com/collections/guitars
[5]
Fret Zealot. 2017. Fret Zealot. (Dec 2017). http://fretzealot.com/
[6]
Tobias Grosshauser, and Gerhard Tröster. 2013. Finger Position and Pressure Sensing Techniques for String and Keyboard Instruments. In Proceedings of the international conference on New Interfaces for Musical Expression (NIME '13), 479--484. http://nime.org/proceedings/2013/nime2013_286.pdf
[7]
Tobias Grosshauser, and Gerhard Tröster. 2013. Optical bow position, speed and contact point detection. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication (UbiComp '13), 211--214.
[8]
Tobias Grosshauser, and Gerhard Tröster. 2014. Musical instrument interaction: development of a sensor fingerboard for string instruments. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI '14), 177--180.
[9]
Robert H. Jack, Tony Stockman, and Andrew McPherson. 2016. Effect of latency on performer interaction and subjective quality assessment of a digital musical instrument. In proceeding of the Audio Mostly (AM '16), 116--123.
[10]
Andrew McPherson. 2012. TouchKeys: Capacitive Multi-Touch Sensing on a Physical Keyboard. In proceedings of the 12th international conference on New Interfaces for Musical Expression (NIME '12), 195--198. http://nime.org/proceedings/2012/nime2012_195.pdf
[11]
NSERLAB. 2017. Real-time Recognition of Guitar Performance Using Two Sensor Groups for Interactive Lesson. Video. (November 28, 2017). Retrieved November 28, 2017 from https://youtu.be/dpJqdGgdPVc
[12]
Laurel S. Pardue, Christopher Harte, and Andrew P. McPherson. 2015. A Low-Cost Real-Time Tracking System for Violin. Journal of New Music Research. 44, 4: 305--323.
[13]
Soonuk Seol, Yejin Shin, and Kangwon Lee. 2016. Learning Guitar with an Embedded System. Contemporary Engineering Sciences. 9, 12: 553--560.
[14]
Fabian-Robert Stöter, Michael Müller, and Bernd Edler. 2015. Multi-Sensor Cello Recordings for Instantaneous Frequency Estimation. In Proceedings of the 23rd ACM international conference on Multimedia (MM '15), 995--998.
[15]
Nick Summers. 2014. gTar: This iPhone-powered guitar isn't the educational instrument you've been dreaming of. (January 25, 2014). Retrieved August 2, 2017 from http://tnw.to/q3Fnh
[16]
Tim Vets, Jonas Degrave, Luc Nijs, Federica Bressan, and Marc Leman. 2017. PLXTRM: Prediction-Led eXtended-guitar Tool for Real-time Music applications and live performance. Journal of New Music Research. 46, 2: 187--200.
[17]
Kazuki Yazawa, Daichi Sakaue, Kohei Nagira, Katsutoshi Itoyama, and Hiroshi G. Okuno. 2013. Audio-based guitar tablature transcription using multipitch analysis and playability constraints. In Acoustics, Speech and Signal Processing (ICASSP '13), 196--200.
[18]
Jun Yin, Ye Wang, and David Hsu. 2005. Digital violin tutor: An integrated system for beginning violin learners. In proceedings of the 13th annual ACM international conference on Multimedia, 976--985.
[19]
Bingjun Zhang, Jia Zhu, Ye Wang, and Wee Kheng Leow. 2007. Visual analysis of fingering for pedagogical violin transcription. In Proceedings of the 15th ACM international conference on Multimedia (MM '07), 521--524.

Cited By

View all
  • (2023)PressurePick: Muscle Tension Estimation for Guitar Players Using Unobtrusive Pressure SensingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606742(1-11)Online publication date: 29-Oct-2023
  • (2023)A Comparison of Interfaces for Learning How to Play a Mixed Reality Handpan2023 IEEE Gaming, Entertainment, and Media Conference (GEM)10.1109/GEM59776.2023.10390457(1-6)Online publication date: 19-Nov-2023
  • (2022)Evaluating Electric Guitar Strumming Form as Musically Correct Rhythm And Sharpness Using Wrist-Wom Inertial Motion-Tracking Device2022 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE53296.2022.9730320(1-6)Online publication date: 7-Jan-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
TEI '18: Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction
March 2018
763 pages
ISBN:9781450355681
DOI:10.1145/3173225
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automatic note inference
  2. fingering position detection
  3. guitar
  4. music transcription
  5. performance feedback
  6. sensors

Qualifiers

  • Research-article

Funding Sources

  • Small and Medium Business Administration

Conference

TEI '18
Sponsor:

Acceptance Rates

TEI '18 Paper Acceptance Rate 37 of 130 submissions, 28%;
Overall Acceptance Rate 393 of 1,367 submissions, 29%

Upcoming Conference

TEI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)PressurePick: Muscle Tension Estimation for Guitar Players Using Unobtrusive Pressure SensingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606742(1-11)Online publication date: 29-Oct-2023
  • (2023)A Comparison of Interfaces for Learning How to Play a Mixed Reality Handpan2023 IEEE Gaming, Entertainment, and Media Conference (GEM)10.1109/GEM59776.2023.10390457(1-6)Online publication date: 19-Nov-2023
  • (2022)Evaluating Electric Guitar Strumming Form as Musically Correct Rhythm And Sharpness Using Wrist-Wom Inertial Motion-Tracking Device2022 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE53296.2022.9730320(1-6)Online publication date: 7-Jan-2022
  • (2022)Interactive Training System for Electric Guitar Strumming Form by Using Inertial Motion Sensors2022 IEEE 11th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE56475.2022.10014279(97-100)Online publication date: 18-Oct-2022
  • (2021)Investigating Page Turning Methods for Sheet Music during Piano PlayAdjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447527.3474863(1-6)Online publication date: 27-Sep-2021
  • (2021)Let’s Frets! Assisting Guitar Students During Practice via Capacitive SensingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445595(1-12)Online publication date: 6-May-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media