Skip to main content
SpringerLink
Log in

Continuous query processing over music streams based on approximate matching mechanisms

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

It is foreseen that more and more music objects in symbolic format and multimedia objects, such as audio, video, or lyrics, integrated with symbolic music representation (SMR) will be published and broadcasted via the Internet. The SMRs of the flowing songs or multimedia objects will form a music stream. Many interesting applications based on music streams, such as interactive music tutorials, distance music education, and similar theme searching, make the research of content-based retrieval over music streams much important. We consider multiple queries with error tolerances over music streams and address the issue of approximate matching in this environment. We propose a novel approach to continuously process multiple queries over the music streams for finding all the music segments that are similar to the queries. Our approach is based on the concept of n-grams, and two mechanisms are designed to reduce the heavy computation of approximate matching. One mechanism uses the clustering of query n-grams to prune the query n-grams that are irrelevant to the incoming data n-gram. The other mechanism records the data n-gram that matches a query n-gram as a partial answer and incrementally merges the partial answers of the same query. We implement a prototype system for experiments in which songs in the MIDI format are continuously broadcasted, and the user can specify musical segments as queries to monitor the music streams. Experiment results show the effectiveness and efficiency of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Altschul, S., Gish, W., Miller, W., Myers, E.W., Lipman, D.J.: Basic local alignment search tool. J. Mol. Biol. (1990)

  2. Avnur, R., Hellerstein, J.M.: Eddies: continuous adaptive query processing. In: Proceedings of ACM SIGMOD International Conference on Management Data (2000)

  3. Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The R*-tree: An efficient and robust access method for points and rectangles. In: Proceedings of ACM SIGMOD International Conference on Management of Data (1990)

  4. Bellini, P., Barthelemy, J., Nesi, P., Zoia, G.: A proposal for the integration of symbolic music notation into multimedia frameworks. In: Proceedings of International Conference on the Web Delivering of Music (2004)

  5. Bellini, P., Nesi, P., Zoia, G.: Symbolic music representation in MPEG. IEEE Multimedia (2005)

  6. Burges, C.J.C., Platt, J.C., Jana, S.: Distortion discriminant analysis for audio fingerprinting. IEEE Trans. on Speech Audio Process. (2003)

  7. Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring streams—a new class of data management applications. In: Proceedings of International Conference on Very Large Data Bases (2002)

  8. Cano, P., Batlle, E., Kalker, T., Haitsma, J.: A review of algorithms for audio fingerprinting. In: International Workshop on Multimedia Signal Processing (2002)

  9. Cano, P., Batlle, E., Mayer, H., Neuschmied, H.: Robust sound modeling for song detection in broadcast audio. In: Processing of International Conference on AES (2002)

  10. Chandrasekaran, S., Franklin, M.J.: Streaming Queries over Streaming Data. In: Proceedings of International Conference on Very Large Data Bases (2002)

  11. Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagraCQ: a scalable continuous query system for internet databases. In: Proceedings of ACM SIGMOD International Conference on Management of Data (2000)

  12. Clausen, M., Engelbrecht, R., Meyer, D., Schmitz, J.: Proms: a web-based tool for searching in polyphonic music. In: Proceedings of International Symposium on Music Information Retrieval (2000)

  13. Cranor, C., Gao, Y., Johnson, T., Shkapenyuk, V., Spatscheck, O.: Gigascope: high performance network monitoring with an SQL interface. In: Proceedings of ACM SIGMOD Conference on Management of Data (2003)

  14. Crochemore, M., Rytter, W.: Text Algorithms. Oxford University Press, Oxford (1994)

    MATH  Google Scholar 

  15. Dannenberg, R.B., Hu, N.: Pattern discovery techniques for music audio. In: Proceedings of International Symposium on Music Information Retrieval (2002)

  16. Dovey, M.J.: A technique for “regular expression” style searching in polyphonic music. In: Proceedings of International Symposium on Music Information Retrieval (2001)

  17. Downie, S., Nelson, M.: Evaluation of a simple and effective music information retrieval method. In: Proceedings of ACM International Conference on Research and Development in Information Retrieval (2000)

  18. Faloutsos, C., Ranganathan, M., Manolopoulos, Y.: Fast subsequence matching in time-series databases. In: Proceedings of ACM SIGMOD International Conference on Management of Data (1994)

  19. Ghias, A., Logan, J., Chamberlin, D., Smith, B.C.: Query by humming: music information retrieval in an audio database. In: Proceedings of ACM Multimedia (1995)

  20. Golab, L., Özsu, M.T.: Issues in data stream management. ACM SIGMOD Record (2003)

  21. Gusfield, D.: Algorithms on Strings, Trees, and Sequences. Cambridge University Press, Cambridge (1997)

    MATH  Google Scholar 

  22. Haitsma, J., Kaller, T., Oostveen, J.: Robust audio hashing for content identification. In: Proceedings of IEEE International Workshop on Content-based Multimedia Indexing (2001)

  23. Haitsma, J., Kalker, T.: A highly robust audio fingerprinting system. In: Proceedings of International Symposium on Music Information Retrieval (2002)

  24. Hellerstein, J., Franklin, M., Chandrasekaran, S., Deshpande, A., Hildrum, K., Madden, S., Raman, V., Shah, M.A.: Adaptive query processing: technology in evolution. IEEE Data Eng. Bull. (2000)

  25. Hsu, J.L., Chen, A.L.P.: Building a platform for performance study of various music information retrieval approaches. In: Proceedings of International Symposium on Music Information Retrieval (2001)

  26. Hsu, J.L., Liu, C.C., Chen, A.L.P.: Discovering non-trivial repeating patterns in music data. IEEE Trans. Multimedia (2001)

  27. Kahveci, T., Singh, A.K.: An efficient index structure for string databases. In: Proceeding of International Conference on Very Large Data Bases (2001)

  28. Kageyama, T., Mochizuki, K., Takashima, Y.: Melody retrieval with humming. In: Proceedings of International Computer Music Conference (1993)

  29. Kang, J., Naughton, J.F., Viglas, S.D.: Evaluating window joins over unbounded streams. In: Proceedings of IEEE International Conference on Data Engineering (2003)

  30. Kornstadt, A.: Themefinder: A Web-Based Melodic Search Tool. In: Computing in Musicology, vol. 11. MIT Press, Cambridge (1998)

  31. Lemstrõm, K., Perttu, S.: SEMEX – An efficient music retrieval prototype. In: Proceedings of International Symposium on Music Information Retrieval (2000)

  32. Liu, N.H., Wu, Y.H., Chen, A.L.P.: Efficient kNN Search in Polyphonic Music Databases Using a Lower Bounding Mechanism. ACM Multimedia Syst. J. (2005)

  33. Liu, N.H., Wu, Y.H., Chen, A.L.P.: An efficient approach to extracting approximate repeating patterns in music databases. In: Proceedings of International Conference on Database Systems for Advanced Applications (2005)

  34. Madden, S., Shah, M., Hellerstein, J.M., Raman, V.: Continuous adaptive continuous queries over streams. In: Proceedings of ACM SIGMOD International Conference on Management Data (2002)

  35. McNab, R.J., Smith, L.A., Witten, I.H., Henderson, C.L., Cunningham, S.J.: Towards the digital music library: tune retrieval from acoustic input. In: Proceedings of Digital Libraries Conference (1996)

  36. McNab, R.J., Smith, L.A., Bainbridge, D., Witten, I.H.: The new zealand digital library MELodyinDEX. Digi. Libr. Mag. (1997)

  37. MIDI Manufacturers Association, Los Angeles, California: The Complete Detailed MIDI 1.0 Specification (1996)

  38. Mongeau, M., Sankoff, D.: Comparison of Musical Sequences. In: Computer and the Humanities (1990)

  39. Moon, Y.S., Whang, K.Y., Han, W.S.: General match: a subsequence matching method in time-series databases based on generalized windows. In: Proceedings of ACM SIGMOD International Conference on Management of Data (2002)

  40. Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G., Olston, C., Rosenstein, J., Varma, R.: Query processing, approximation, and resource management in a data stream management system. In: Proceedings of Biennial Conference on Innovative Data Systems Research (2003)

  41. MusicXML, http://www.recordare.com/

  42. Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. (2001)

  43. Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. Mol. Biol. (1970)

  44. Park, S., Chu, W.W., Yoon, J., Hsu, C.: Efficient searches for similar subsequences of different lengths in sequence databases. In: Proceedings of International Conference on Data Engineering (2000)

  45. Pearson, W.R., Lipman, D.J.: Improved tools for biological sequence comparison. In: Proc. Nat. Acad. Sci. (1998)

  46. Pickens, J.: A survey of feature selection techniques for music information retrieval. In: Proceedings of International Symposium on Music Information Retrieval (2001)

  47. Pirkola, A., Keskustalo, H., Leppanen, E., Kansala, A.P., Jarvelin, K.: Targeted s-gram matching: a novel n-gram matching technique for cross- and monolingual word form variants. Inf. Res. (2002)

  48. Rafiei, D., Mendelzon, A.: Similarity-based queries for time series data. In: Proceedings of ACM SIGMOD International Conference on Management of Data (1997)

  49. Robertson, A.M., Willett, P.: Applications of n-grams in textual information systems. J. Doc. (1998)

  50. Root Cause Analysis Handbook: A Guide to Effective Incident Investigation, by Risk and Reliability Division, ABS Group, Inc. Root Cause Map, ABSG Consulting Inc. (1999)

  51. Salton, G.: Automatic Text Processing: the Transformation, Analysis, and Retrieval of Information by Computer. Addison Wesley, Reading (1989)

    Google Scholar 

  52. Selfridge-Field, E.: Conceptual and representational issues in melodic comparison. Melodic similarity: concepts, procedures, and applications (Computing in Musiccology:11), The MIT Press, Cambridge (1998)

  53. Uitdenbogerd, A., Zobel, J.: Melodic Matching Techniques for Large Music Databases. In: Proceedings of ACM Multimedia (1999)

  54. Ukkonen, E.: Algorithms for approximate string matching. Inf. Control (1985)

  55. Wu, S., Manber, U.: Fast text searching allowing errors. Commun. ACM (1992)

  56. Yao, Y., Gehrke, J.: Query Processing for Sensor Networks. In: Proceedings of Conference on Innovative Data Systems Research (2003)

  57. Zobel, J., Dart, P.: Finding approximate matches in large lexicons. In: Software—practice and experiences (1995)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, National Tsing Hua University, Hsinchu, 30013, Taiwan, ROC

    Hung-Chen Chen & Yu-Chi Soo

  2. Department of Information and Computer Engineering, Chung Yuan Christian University, Chung Li, 32023, Taiwan, ROC

    Yi-Hung Wu

  3. Department of Computer Science, National Chengchi University, Taipei, 11605, Taiwan, ROC

    Arbee L. P. Chen

Authors
  1. Hung-Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Hung Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Chi Soo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arbee L. P. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arbee L. P. Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, HC., Wu, YH., Soo, YC. et al. Continuous query processing over music streams based on approximate matching mechanisms. Multimedia Systems 14, 51–70 (2008). https://doi.org/10.1007/s00530-007-0111-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-007-0111-z

Keywords

Search

Navigation