skip to main content
10.1145/3678299.3678329acmotherconferencesArticle/Chapter ViewAbstractPublication PagesamConference Proceedingsconference-collections
research-article
Open access

Real-Time Pattern Recognition of Symbolic Monophonic Music

Published: 18 September 2024 Publication History

Abstract

This paper investigates the real-time detection of predefined monophonic patterns from the MIDI output of a digital musical instrument. This enables the development of instruments and systems for live music, which can recognize when a musician plays a certain phrase and repurpose such information to trigger external peripherals connected to the instrument. Specifically, we compare the recognition performance of Dynamic Time Warping and Recurrent Neural Network-based approaches. We employ different representation formats of musical data to optimize the efficiency of each computational method. To evaluate the algorithms, a novel dataset is introduced which includes recordings from 20 keyboard players and 20 guitar players. The evaluation focuses on the algorithms’ ability to recognize patterns amid variations that impede a straightforward one-to-one comparison. The results reveal that both methods perform well in detecting up to 3 distinct patterns. However, as the number of different patterns increases up to 10, dynamic time warping exhibits a negative correlation with the recognition performance, while the recurrent neural network maintains high detection accuracy of approximately 98%. Taken together, our findings demonstrate the potential of machine learning in handling complex musical patterns in real-time, paving the way for novel applications involving smart musical instruments.

References

[1]
2018. 2017:Discovery of Repeated Themes & Sections. Retrieved Aug 12, 2024 from https://www.music-ir.org/mirex/wiki/2017:Discovery_of_Repeated_Themes_%26_Sections
[2]
2024. Official MIDI Specifications: General MIDI 1. Retrieved Aug 12, 2024 from https://midi.org/general-midi
[3]
M. Amerotti, S. Benford, B. Sturm, and C. Vear. 2023. A Live Performance Rule System Informed by Irish Traditional Dance Music. In Proceedings of the International Symposium on Computer Music Multidisciplinary Research.
[4]
A. Arronte Alvarez and F. Gómez. 2021. Motivic Pattern Classification of Music Audio Signals Combining Residual and LSTM Networks. International Journal of Interactive Multimedia and Artificial Intelligence (05 2021).
[5]
S. Chakrabarty, R. Islam, E. Pricop, and H. Sarma. 2022. An Approach to Discover Similar Musical Patterns. IEEE Access 10 (2022), 47322–47339.
[6]
T. P. Chen and L. Su. 2017. Discovery of repeated themes and sections with patern clustering. In Presented at the 18th International Society for Music Information Retrieval Conference.
[7]
T. Collins, J. Thurlow, R. Laney, A. Willis, and P. Garthwaite. 2010. A comparative evaluation of algorithms for discovering translational patterns in Baroque keyboard works. In Proceedings of the International Symposium on Music Information Retrieval.
[8]
D. Conklin and C. Anagnostopoulou. 2001. Representation and Discovery of Multiple Viewpoint Patterns. In Proceedings of the 2001 International Computer Music Conference. 479–485.
[9]
R. Dannenberg and N. Hu. 2002. Linear time for discovering non-trivial repeating patterns in music databases. In ISMIR 2002 Conference Proceedings: Third International Conference on Music Information Retrieval. 63–70.
[10]
T. de Reuse and I. Fujinaga. 2019. Pattern Clustering in Monophonic Music by Learning a Non-Linear Embedding From Human Annotations. In Proceedings of the 20th International Society for Music Information Retrieval Conference. 761–768.
[11]
C. Finkensiep, M. Neuwirth, and M. Rohrmeier. 2018. Generalized Skipgrams for Pattern Discovery in Polyphonic Streams. In Proceedings of the 19th International Society for Music Information Retrieval Conference. 23–27.
[12]
F Gómez, M Tizón, A Arronte Alvarez, and V Padilla. 2022. Rhetorical Pattern Finding. International Journal of Interactive Multimedia and Artificial Intelligence (11 2022).
[13]
J. L. Hsu, C. Liu, and A. Chen. 2001. Discovering nontrivial repeating patterns in music data. IEEE Trans. Multim. 3 (2001), 311–325.
[14]
I. Knopke and F. Jürgensen. 2009. A System for Identifying Common Melodic Phrases in the Masses of Palestrina. Journal of New Music Research 38, 2 (2009), 171–181.
[15]
Q. Kong, Y. Cao, T. Iqbal, Y. Wang, W. Wang, and M. D. Plumbley. 2020. PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2020), 2880–2894.
[16]
O. Lartillot and P. Toiviainen. 2007. Motivic matching strategies for automated pattern extraction. Musicae Scientiae 11, 1_suppl (2007), 281–314.
[17]
C. Meek. 2003. Automatic Thematic Extractor. Journal of Intelligent Information Systems 21 (2003), 9–33.
[18]
O. Melkonian, I. Ren, W. Swierstra, and A. Volk. 2019. What Constitutes a Musical Pattern?. In Proceedings of the 7th ACM SIGPLAN International Workshop on Functional Art, Music, Modeling, and Design (Berlin, Germany). Association for Computing Machinery, New York, NY, USA, 95–105.
[19]
D. Meredith. 2017. Using SIATECCOMPRESS to discover repeated themes and sections in polyphonic music. In Presented at the 18th International Society for Music Information Retrieval Conference.
[20]
D. Meredith, K. Lemström, and G. A. Wiggins. 2002. Algorithms for discovering repeated patterns in multidimensional representations of polyphonic music. Journal of New Music Research 31, 4 (2002), 321–345.
[21]
B. C. J. Moore. 2012. An introduction to the psychology of hearing. Brill.
[22]
O. Nieto and M. M. Farbood. 2017. Music segmentation techniques and greedy path finder algorithm to discover musical patterns. In Presented at the 18th International Society for Music Information Retrieval Conference.
[23]
M. Pesek, A. Leonardis, and M. Marolt. 2017. SYMCHMMERGE: an extension to the compositional hierarchial model for pattern discovery in symbolic music representations. In Presented at the 18th International Society for Music Information Retrieval Conference.
[24]
I. Y. Ren, H. V. Koops, A. Volk, and W. Swierstra. 2017. In Search of the Consensus Among Musical Pattern Discovery Algorithms. In Proceedings of the 18th International Society for Music Information Retrieval Conference. 23,27.
[25]
I. Shmulevich, O. Yli-Harja, E. Coyle, D. Povel, and K. Lemström. 2001. Perceptual Issues in Music Pattern Recognition: Complexity of Rhythm and Key Finding. Computers and the Humanities 35, 1 (2001), 23–35.
[26]
M. Shokoohi-Yekta, B. Hu, H. Jin, J. Wang, and E. Keogh. 2017. Generalizing DTW to the multi-dimensional case requires an adaptive approach. Data Mining and Knowledge Discovery 31 (01 2017). https://doi.org/10.1007/s10618-016-0455-0
[27]
N. Silva, C. Fischione, and L. Turchet. 2020. Towards Real-Time Detection of Symbolic Musical Patterns: Probabilistic vs. Deterministic Methods. In 2020 27th Conference of Open Innovations Association (FRUCT). 238–246.
[28]
N. Silva and L. Turchet. 2022. A Structural Similarity Index bsed method to detect symbolic monophonic patterns in real-time. In Proceedings of the 25th International Conference on Digital Audio Effects. 161–168.
[29]
M. Thomas, Y. V. S. Murthy, and S. G. Koolagudi. 2016. Detection of largest possible repeated patterns in Indian audio songs using spectral features. In 2016 IEEE Canadian Conference on Electrical and Computer Engineering. 1–5.
[30]
L. Turchet. 2019. Smart Musical Instruments: vision, design principles, and future directions. IEEE Access 7 (2019), 8944–8963.
[31]
L. Turchet, C. Fischione, G. Essl, D. Keller, and M. Barthet. 2018. Internet of Musical Things: Vision and Challenges. IEEE Access 6 (2018), 61994–62017.
[32]
P. van Kranenburg, A. Volk, and F. Wiering. 2013. A Comparison between Global and Local Features for Computational Classification of Folk Song Melodies. Journal of New Music Research 42, 1 (2013), 1–18.
[33]
A. Volk and P. van Kranenburg. 2012. Melodic similarity among folk songs: An annotation study on similarity-based categorization in music. Musicae Scientiae 16, 3 (2012), 317–339. https://doi.org/10.1177/1029864912448329
[34]
C. I. Wang, J. Hsu, and S. Dubnov. 2015. Music Pattern Discovery with Variable Markov Oracle: A Unified Approach to Symbolic and Audio Representations. In Proceedings of the 16th International Society for Music Information Retrieval Conference.
[35]
M. Wöllmer, M. Al-Hames, F. Eyben, B. Schuller, and G. Rigoll. 2009. A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams. Neurocomputing 73, 1 (2009), 366–380.

Cited By

View all
  • (2024)DSP as a Service: Foundations and DirectionsIEEE Open Journal of the Communications Society10.1109/OJCOMS.2024.34646965(6212-6226)Online publication date: 2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AM '24: Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures
September 2024
565 pages
ISBN:9798400709685
DOI:10.1145/3678299
This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2024

Check for updates

Author Tags

  1. Internet of Musical Things
  2. Real-Time Pattern Recognition
  3. Smart Musical Instruments

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AM '24

Acceptance Rates

Overall Acceptance Rate 177 of 275 submissions, 64%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)133
  • Downloads (Last 6 weeks)38
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)DSP as a Service: Foundations and DirectionsIEEE Open Journal of the Communications Society10.1109/OJCOMS.2024.34646965(6212-6226)Online publication date: 2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media