Abstract
The traditional ways to learn musical instruments are having lessons with teachers, or self-paced by teaching videos and books. Nowadays, the musical instrument self-learning systems are in the ascendant. However, it is difficult for a musical instrument learner to know if his playing posture is correct unless the teacher provides guidance in person and corrects it immediately. Due to the breakthrough development of computer vision technology, computers with camera lenses can act as teachers’ eyes to detect the learners’ movements and give some assistance for their self-learning. Erhu, a kind of Chinese traditional stringed instrument, is getting popular worldwide. In this research, taking the erhu bowing method as an example, we use YOLO object detection technology to track the trajectory of the bow in the video and evaluate whether it meets the level and straightness requirements. For the purpose, we first define the measurement methods of level and straightness and design a system to record the progress. In 2016, Joseph Redmon et al. proposed a real-time object detection method called YOLO, “You Only Look Once”. The multi-layer CNN architecture is executed only once, which greatly improves the detection speed. To track the movement of erhu’s bow, we must at least detect the coordinates of two objects, the erhu bow and the right hand of the player who holding the bow. Our images data set to train the object detection model came from several Chinese music society in Taiwan. The experimental results show that the model can successfully mark the desired objects. Finally, the study found that even senior erhu players still have some deviations, but they are more stable than beginners. Self-correction through system prompts would help learners to reduce the bias problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kikukawa, F., Ishihara, S., Soga, M., Taki, H.: Development of a learning environment for playing erhu by diagnosis and advice regarding finger position on strings. In: Proceedings of the Conference on New Instruments for Musical Expression, pp. 271–276 (2013)
Hsiao, C.P., Li, R., Yan, X., Do, E.Y.: Tactile teacher: sensing finger tapping in piano playing. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 257–260 (2015). https://doi.org/10.1145/2677199.2680554
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016)
Yao, W.: Follow Me to Learn the Erhu. Anhui literature and Art Publishing House, Ch. 2, pp. 13–16 (2011)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing System, pp. 1097–1105 (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. Cornell University. https://arxiv.org/abs/1804.02767. Accessed 8 Apr 2018
Pan, J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Young, D.: The hyperbow controller: real-time dynamics measurement of violin performance. In: Proceedings of the Conference on New Instruments for Musical Expression (NIME) (2002)
Kikukawa, F., Soga, M., Taki, H.: Development of a gesture learning environment for novices’ erhu bow strokes. Procedia Comput. Sci. 35, 1323–1332 (2014). https://doi.org/10.1016/j.procs.2014.08.171
Redmon, J.: Darknet: open source neural networks in C (2013–2016). https://pjreddie.com/darknet/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, B., Dow, CR., Peng, CJ. (2020). Bowing Detection for Erhu Learners Using YOLO Deep Learning Techniques. In: Stephanidis, C., Antona, M. (eds) HCI International 2020 - Posters. HCII 2020. Communications in Computer and Information Science, vol 1225. Springer, Cham. https://doi.org/10.1007/978-3-030-50729-9_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-50729-9_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50728-2
Online ISBN: 978-3-030-50729-9
eBook Packages: Computer ScienceComputer Science (R0)