Skip to main content
SpringerLink
Log in

A Novel Intelligent Assessment Based on Audio-Visual Data for Chinese Zither Fingerings

  • Conference paper
  • First Online:
Image and Graphics (ICIG 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14358))

Included in the following conference series:

  • 302 Accesses

Abstract

In this paper, we make a novel study on the intelligent assessment for Chinese zither (Zheng) fingerings in the cross field of art AI. Due to the gaps between science and art, there is a lack of deep combination of musical instrument expertise with computer aided technology. This paper attempts to explore the inheritance and innovation of Chinese classical musical instrument in this interdisciplinary study. We integrate video understanding, audio analysis with zither professional knowledge to design a pipeline for Zheng intelligent evaluation. Firstly, we establish a Zheng fingering Dataset based on video and audio (ZF-VA-Dataset). Then, according to zither teaching experience and accumulated research on machine perception, we formulate a Zheng fingering assessment scale, which is served as a bridge between Zheng knowledge and audio-visual technology. Finally, an intelligent evaluation method based on dual modes is put forward for Chinese zither fingerings. The experimental results show that the proposed scheme is feasible and effective in achieving intelligent assessment of Zheng fingerings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ostermann, F., Vatolkin, I., Ebeling, M.: AAM: a dataset of artificial audio multitracks for diverse music information retrieval tasks. EURASIP J. Audio, Speech, Music Proc. 2023(1), 1–12 (2023)

    Google Scholar 

  2. Bao, N.: Analysis of music retrieval based on emotional tags environment. J. Environ. Public Health 2022, 4670963 (2022)

    Article  Google Scholar 

  3. Olivieri, M., et al.: Audio information retrieval and musical acoustics. IEEE Instrum. Meas. Mag. 24(7), 10–20 (2021)

    Article  Google Scholar 

  4. Alfaro-Contreras, M., Valero-Mas, J.J., Iñesta, J.M., Calvo-Zaragoza, J.: Late multimodal fusion for image and audio music transcription. Expert Syst. Appl. 216, 119491 (2023)

    Article  Google Scholar 

  5. Román, M.A., Pertusa, A., Calvo-Zaragoza, J.: Data representations for audio-to-score monophonic music transcription. Expert Syst. Appl. 162, 113769 (2020)

    Article  Google Scholar 

  6. Shibata, K., Nakamura, E., Yoshii, K.: Non-local musical statistics as guides for audio-to-score piano transcription. Inf. Sci. 566, 262–280 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  7. Li, H., Sakai, T., Tanaka, A., Ogura, M., Lee, C., Yamaguchi, S., Imazato, S.: Interpretable AI explores effective components of CAD/CAM resin composites. J. Dent. Res. 101(11), 1363–1371 (2022)

    Article  Google Scholar 

  8. Deng, Y., Xu, Z., Zhou, L., Liu, H., Huang, A.: Research on AI composition recognition based on music rules. Lect. Notes Electr. Eng. 761, 187–197 (2021)

    Article  Google Scholar 

  9. Alabduljabbar, H., Khan, K., Awan, H.H., Alyousef, R., Mohamed, A.M., Eldin, S.M.: Modeling the capacity of engineered cementitious composites for self-healing using AI-based ensemble techniques. Case Stud. Constr. Mater. 18, e01805 (2023)

    Google Scholar 

  10. Vallejos, S., Da Rocha, L., Araujo, G.R., Berdun, L., Toscani, R.: Preference-based AI planning for web service composition. IEEE Latin Am. Trans. 18(11), 1987–1995 (2020)

    Article  Google Scholar 

  11. Yang, R.: Extended QUALIFLEX method for electronic music acoustic quality evaluation based on the picture fuzzy multiple attribute group decision making. J. Intell. Fuzzy Syst. 44(3), 5021–5032 (2023)

    Article  Google Scholar 

  12. Guo, T.: Evaluation of the emotion model in electronic music based on PSO-BP. Comput. Intell. Neurosci. 2022(1), 1–9 (2022)

    Google Scholar 

  13. de Marques Andrade, T.G.: Free robotics and the teaching of physics and programming developing an electronic music keyboard (Article). Texto Livre 11(3), 317–330 (2018)

    Google Scholar 

  14. Hanson, A.: Music Keyboard Based on Flexible Hybrid Electronics (2018)

    Google Scholar 

  15. Sparkes, M.: Being graded by an AI could improve your piano playing. New Scientist 249(3319), 15 (2021)

    Article  Google Scholar 

  16. Chen, Y., Zheng, N.: AI based research on exploration and innovation of development direction of piano performance teaching in university. J. Intell. Fuzzy Syst. 1–7 (2020)

    Google Scholar 

  17. Guo, R., Cui, J., Zhao, W., Li, S.: AI and AR based interface for piano training. In: 2020 International Conference on Virtual Reality and Visualization (ICVRV) (2020)

    Google Scholar 

  18. Parmar, P., Reddy, J., Morris, B.: Piano skills assessment. In: 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP) (2021)

    Google Scholar 

  19. Arthur, P., McPhee, E., Blom, D.: Determining what expert piano sight-readers have in common. Music Educ. Res. 22(4), 447–456 (2020)

    Article  Google Scholar 

  20. Zheng, Y.D., Tian, T., Zhang, A.: Training strategy of music expression in piano teaching and performance by intelligent multimedia technology. Int. Trans. Electr. Energy Syst. 2022, 1–14 (2022)

    Google Scholar 

  21. Giraldo, S., et al.: Automatic assessment of tone quality in violin music performance. Front. Psychol. 10, 334 (2019)

    Article  Google Scholar 

  22. Nauncef, A.M., et al.: 14. Interdisciplinary approach to assessing the acoustic quality of violins. Rev. Artistic Educ. 23(1), 111–119 (2022)

    Article  Google Scholar 

  23. Giampiccolo, R., Bernardini, A., Sarti, A.: Virtualization of guitar pickups through circuit inversion. IEEE Signal Process. Lett. 30, 1–5 (2023)

    Article  Google Scholar 

  24. Ferretti, S.: Clustering of musical pieces through complex networks: an assessment over guitar solos. IEEE Multimedia 25, 57–67 (2018)

    Article  Google Scholar 

  25. Shen, D.: Sound design of guqin culture: interactive art promotes the sustainable development of traditional culture. Sustainability 14(4), 2356 (2022)

    Article  MathSciNet  Google Scholar 

  26. Zhang, M., Zhang, J.: A gesturally controlled virtual musical instruments for Chinese Guqin. In: International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI) (2020)

    Google Scholar 

  27. Zhao, W., et al.: CZ-Base: a database for hand gesture recognition in Chinese zither intelligence education. In: Zhai, G., Zhou, J., Yang, H., An, P., Yang, X. (eds.) Digital TV and Wireless Multimedia Communication: 17th International Forum, IFTC 2020, Shanghai, China, December 2, 2020, Revised Selected Papers, pp. 282–292. Springer Singapore, Singapore (2021). https://doi.org/10.1007/978-981-16-1194-0_25

    Chapter  Google Scholar 

  28. Zhao, W., Wang, S., Zhao, Y., Wei, J., Li, T., Zhang, R.: Self-designed hierarchical network based hand shape intelligent recognition and evaluation for Chinese zither performing. Displays 76, 102291 (2023)

    Article  Google Scholar 

  29. Zhao, W., Wang, S., Zhao, Y., Wei, J., Li, T.: Knowledge and data co-driven intelligent assessment of Chinese zither fingerings. Displays 78, 102442 (2023)

    Article  Google Scholar 

  30. Zhang, F., Bazarevsky, V., et al.: Mediapipe hands: on-device real-time hand tracking. CVPR (2020)

    Google Scholar 

Download references

Acknowledgment

This work is supported by Science and Technology Development Plan of Jilin Province (No.20210201027GX); National Natural Science Foundation of China (No.61901187); Science and Technology Development Plan of Jilin Province (No.20220101104JC); Fundamental Research Funds for the Central Universities (No.415010300076).

Author information

Authors and Affiliations

  1. College of Communication Engineering, Jilin University, Changchun, China

    Wenting Zhao, Shigang Wang, Yan Zhao, Jian Wei & Tianshu Li

Authors
  1. Wenting Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tianshu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shigang Wang .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Huchuan Lu

  2. University of Sydney, Sydney, NSW, Australia

    Wanli Ouyang

  3. Shenzhen University, Shenzhen, China

    Hui Huang

  4. Tsinghua University, Beijing, China

    Jiwen Lu

  5. Dalian University of Technology, Dalian, China

    Risheng Liu

  6. Institute of Automation, CAS, Beijing, China

    Jing Dong

  7. University of Technology Sydney, Sydney, NSW, Australia

    Min Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, W., Wang, S., Zhao, Y., Wei, J., Li, T. (2023). A Novel Intelligent Assessment Based on Audio-Visual Data for Chinese Zither Fingerings. In: Lu, H., et al. Image and Graphics . ICIG 2023. Lecture Notes in Computer Science, vol 14358. Springer, Cham. https://doi.org/10.1007/978-3-031-46314-3_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46314-3_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46313-6

  • Online ISBN: 978-3-031-46314-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation