skip to main content
10.1145/2435264.2435305acmconferencesArticle/Chapter ViewAbstractPublication PagesfpgaConference Proceedingsconference-collections
research-article

An FPGA based parallel architecture for music melody matching

Published: 11 February 2013 Publication History

Abstract

We propose an FPGA-based high performance parallel architecture for music retrieval through singing. The database consists of monophonic MIDI files which are modeled into strings, and the user sung query is modeled as a set of regular expressions (regexp), with consideration of possible key transpositions and tempo variations to tolerate imperfectly sung queries. An approximate regexp matching algorithm is developed to calculate the similarity between a regexp and a string, using edit distance as the metrics. The algorithm supports user sung queries starting anywhere in the database song, not necessarily from the beginning. Using the proposed formal models and algorithms, the similarity between the user sung query and each song in the database can be evaluated and the top-10 most similar results will be reported. We designed the approximate regexp matching algorithm in such way that all terms of the regexp can execute concurrently, which perfectly fits the massive parallelism provided by FPGA. The FPGA implemented melody matching engine (MME) is a parameterized modular architecture that can be reconfigured to implement different regexps by simply updating their parameter registers, and can therefore avoid the time-consuming code re-synthesis. MME also includes an on-board DDR2 memory to store the database, so that they can be read in to calculate edit distances locally on the board. This way, each MME forms a self-contained system and multiple MMEs can be clustered to increase parallel processing power, with virtually no overhead. MME is evaluated using the query corpus of ThinkIT with 355 sung files and database of 5563 MIDI files. It achieves a top-10 hit rate of 90.7% and a runtime of 19.4 seconds, averaging 54.6 milliseconds for a single query. MME achieves significant speedup over software-based systems while providing the same level of flexibility.

References

[1]
http://en.wikipedia.org/wiki/Query_by_humming
[2]
M. A. Casey, R. Veltkamp, M. Goto, M. Leman, C. Rhodes, and M. Slaney, "Content-Based Music Information Retrieval: Current Directions and Future Challenges", Proceedings of the IEEE, 2008.
[3]
R. Typke, F. Wiering, and R. C. Veltkamp, "A Survey of Music Information Retrieval Systems", ISMIR 2005.
[4]
N. Orio, "Music Retrieval: A Tutorial and Review", Foundations and Trends® in Information Retrieval, Vol. 1, No. 1, pp 1--90, 2006.
[5]
J. Pickens, "Feature Selection for Polyphonic Music Retrieval", SIGIR 2001.
[6]
J. Pickens, "Harmonic Models for Polyphonic Music Retrieval", CIKM 2002.
[7]
S. Doraisamy and S. M. R üger, "An Approach Towards a Polyphonic Music Retrieval System", ISMIR 2001.
[8]
http://en.wikipedia.org/wiki/MIDI
[9]
L. R. Rabiner, "On the Use of Autocorrelation Analysis for Pitch Detection", IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. ASSP-25, No. 1, 1977
[10]
A. d. Cheveigne and H. Kawahara, "YIN, a Fundamental Frequency Estimator for Speech and Music", The Journal of the Acoustical Society of America, 2002
[11]
P. McLeod and G. Wyvill, "A Smart Way to Find Pitch", ICMC 2005.
[12]
A. M. Noll, "Cepstrum Pitch Determination," Journal of the Acoustical Society of America, Vol. 41, No. 2, 1967
[13]
A. Ghias, J. Logan, D. Chamberlin, and B. Smith, "Query By Humming: Musical Information Retrieval In an Audio Database", Proc of ACM Multimedia, 1995
[14]
C.-C. Liu, J.-L. Hsu, and A. L.P. Chen, "An Approximate String Matching Algorithm for Content-Based Music Data Retrieval", ICMCS 1999.
[15]
K. Lemström, "String Matching Techniques for Music Retrieval", PhD thesis, University of Helsinki, 2000
[16]
Y.-H. Tseng, "Content-Based Retrieval for Music Collections", SIGIR 1999.
[17]
S. Downie and M. Nelson, "Evaluation of a Simple and Effective Music Information Retrieval System", SIGIR 2000.
[18]
B. Cui, H. V. Jagadish, B. C. Ooi, and K.-L. Tan, "Compacting Music Signatures for Efficient Music Retrieval", EDBT 2008.
[19]
J. Shifrin, B. Pardo, C. Meek, and W. Birmingham, "HMM-Based Musical Query Retrieval," JCDL 2002.
[20]
E. Unal, S. S. Narayanan, and E. Chew, "A Statistical Approach to Retrieval under User-dependent Uncertainty in Query-by-Humming Systems", MIR2004.
[21]
N. H. Adams, M. A. Bartsch, and G. H. Wakefield, "Note Segmentation and Quantization for Music Information Retrieval", IEEE Transactions on Audio, Speech, and Language Processing, Vol. 14, No. 1, 2006
[22]
Levenshtein VI. "Binary codes capable of correcting deletions, insertions, and reversals". Soviet Physics Doklady, 10: 707--10, 1966.
[23]
J.-S. Jang, H.-R. Lee, and M.-Y. Kao, "Content-based Music Retrieval Using Linear Scaling and Branch-and-bound Tree Search", ICME 2001.
[24]
P. Senin, "Dynamic Time Warping Algorithm Review", University of Hawaii at Manoa, Tech. Rep., 2008.
[25]
Y. Zhu and D. Shasha, "Warping Indexes with Envelope Transforms for Query by Humming", SIGMOD 2003.
[26]
X. Wu, M. Li, J. Liu, J. Yang, and Y. Yan, "A Top-down Approach to Melody Match in Pitch Contour for Query by Humming", ISCSLP 2006.
[27]
G. P. Nam, T. T. T. Luong, H. H. Nam, K. R. Park, and S.-J. Park, "Intelligent Query by Humming System Based on Score Level Fusion of Multiple Classifiers", EURASIP Journal on Advances in Signal Processing, 2011.
[28]
J.-S. Jang and H.-R. Lee, "Hierarchical Filtering Method for Content-based Music Retrieval via Acoustic Input", ACM Multimedia, 2001.
[29]
L. Wang, S. Huang, S. Hu, J. Liang, and B. Xu, "An Efficient Method for Query by Humming System Based on Multi-Similarity Measurement Fusion", ICALIP 2008.
[30]
R. B. Dannenberg, W. P. Birmingham, G. Tzanetakis, C. Meek, N. Hu, and B. Pardo, "The MUSART Testbed for Query-By-Humming Evaluation", ISMIR 2003.
[31]
N. Hu and R. B. Dannenberg, "A Comparison of Melodic Database Retrieval Techniques Using Sung Queries", JCDL 2002.
[32]
http://www.fourmilab.ch/webtools/midicsv/
[33]
http://audition.ens.fr/adc/sw/yin.zip
[34]
H. Wang, S. Pu, G. Knezek, and J.-C. Liu, "MIN-MAX: A Counter Based Algorithm for Regular Expression Matching", IEEE Transactions on Parallel and Distributed Systems, January 2013.
[35]
Altera white paper, "Understanding Metastability in FPGAs", http://www.altera.com/literature/wp/wp-01082-quartus-ii-metastability.pdf
[36]
K. Eguro, "SIRC: An Extensible Reconfigurable Computing Communication API", FCCM 2010.
[37]
http://mirlab.org/dataSet/public/IOACAS_QBH_Coprus1.rar
[38]
http://www.esac-data.org/
[39]
http://www.music-ir.org/mirex/wiki/MIREX_HOME
[40]
http://www.music-ir.org/mirex/wiki/2009:Query-by-Singing/Humming_Results
[41]
http://www.music-ir.org/mirex/wiki/2010:Query-by-Singing/Humming_Results
[42]
http://www.music-for-music-teachers.com/happy-birthday.html

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FPGA '13: Proceedings of the ACM/SIGDA international symposium on Field programmable gate arrays
February 2013
294 pages
ISBN:9781450318877
DOI:10.1145/2435264
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 February 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. FPGA
  2. music information retrieval
  3. regular expression

Qualifiers

  • Research-article

Conference

FPGA '13
Sponsor:

Acceptance Rates

Overall Acceptance Rate 125 of 627 submissions, 20%

Upcoming Conference

FPGA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 298
    Total Downloads
  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media