Skip to main content
SpringerLink
Log in

A digital library framework for heterogeneous music collections: from document acquisition to cross-modal interaction

  • Published:
International Journal on Digital Libraries Aims and scope Submit manuscript

Abstract

In this paper, we present a digital library system for managing heterogeneous music collections. The heterogeneity refers to various document types and formats as well as to different modalities, e. g., CD-audio recordings, scanned sheet music, and lyrics. The system offers a full-fledged, widely automated document processing chain: digitization, indexing, annotation, access, and presentation. Our system is implemented as a generic and modular music repository based on a service-oriented software architecture. As a particular strength of our approach, the various documents representing aspects of a piece of music are jointly considered in all stages of the document processing chain. Our user interfaces allow for a multimodal and synchronized presentation of documents (WYSIWYH: what you see is what you hear), a score- or lyrics-based navigation in audio, as well as a cross- and multimodal retrieval. Hence, our music repository may be called a truly cross-modal library system. In our paper, we describe the system components, outline the techniques of the document processing chain, and illustrate the implemented functionalities for user interaction. We describe how the system is put into practice at the Bavarian State Library (BSB) Munich as a part of the German PROBADO Digital Library Initiative (PDLI).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allamanche, E., Herre, J., Fröba, B., Cremer, M.: AudioID: Towards content-based identification of audio material. In: Proceedings of the 110th Audio Engineering Society (AES) Convention (2001)

  2. Arifi V., Clausen M., Kurth F., Müller M.: Synchronization of music data in score-, MIDI- and PCM-format. Comput. Musicol. 13, 9–33 (2004)

    Google Scholar 

  3. Baggi, D., Barate, A., Haus, G., Ludovico, L.A.: NINA—navigating and interacting with notation and audio. In: Proceedings of the 2nd International Workshop on Semantic Media Adaptation and Personalization (SMAP), pp. 134–139. IEEE Computer Society, Washington, DC, USA (2007). doi:10.1109/SMAP.2007.28

  4. Bainbridge, D., Thompson, J., Witten, I.H.: Assembling and enriching digital library collections. In: Proceedings of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), pp. 323–334. IEEE Computer Society, Washington, DC, USA (2003)

  5. Bartsch M.A., Wakefield G.H.: Audio thumbnailing of popular music using chroma-based representations. IEEE Trans. Multimed. 7(1), 96–104 (2005)

    Article  Google Scholar 

  6. Birmingham, W.P., Pardo, B., Meek, C., Shifrin, J.: The MusArt music-retrieval system: an overview. D-Lib Magazine 8(2) (2002). doi:10.1045/february2002birmingham. URL http://www.dlib.org/dlib/february02/birmingham/02birmingham.html

  7. Birmingham, W.P., O’Malley, K., Dunn, J.W., Scherle, R.: V2V: a second variation on query-by-humming. In: Proceedings of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), pp. 380–380. IEEE Computer Society, Washington, DC, USA (2003)

  8. Blümel, I., Krottmaier, H., Wessel, R.: The PROBADO framework: a repository for architectural 3D-models. In: International Conference on Online Repositories in Architecture. Fraunhofer irb Verlag (2008)

  9. Byrd, D., Schindele, M.: Prospects for improving OMR with multiple recognizers. In: Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR), pp. 41–46 (2006)

  10. Cano, P., Battle, E., Kalker, T., Haitsma, J.: A review of audio fingerprinting. In: Proceedings of the 5th IEEE Workshop on Multimedia Signal Processing (MMSP) (2002)

  11. Choudhury, G., DiLauro, T., Droettboom, M., Fujinaga, I., Harrington, B., MacMillan, K.: Optical music recognition system within a large-scale digitization project. In: Proceedings of the 1st International Symposium on Music Information Retrieval (ISMIR) (2000)

  12. Clausen M., Kurth F.: A unified approach to content-based and fault-tolerant music recognition. IEEE Trans. Multimed. 6(5), 717–731 (2004)

    Article  Google Scholar 

  13. D’Aguanno A., Vercellesi G.: Automatic music synchronization using partial score representation based on IEEE 1599. J. Multimed. 4(1), 19–24 (2009)

    Google Scholar 

  14. Damm, D., Kurth, F., Fremerey, C., Clausen, M.: A concept for using combined multimodal queries in digital music libraries. In: Proceedings of the 13th European Conference on Research and Advanced Technology for Digital Libraries (ECDL) (2009)

  15. Damnjanovic, I., Reiss, J., Barry, D.: Enabling access to sound archives through integration, enrichment , and retrieval. In: Proceedings of the 2008 IEEE International Conference on Multimedia and Expo (ICME), pp. 1597–1598 (2008). doi:10.1109/ICME.2008.4607756

  16. Dannenberg, R.B., Raphael, C.: Music score alignment and computer accompaniment. In: Pardo, B. (ed.): Special Issue: Music Information Retrieval, vol. 49, pp. 38–43. ACM, New York, NY, USA (2006). doi:10.1145/1145287.1145311

  17. Diet, J., Kurth, F.: The PROBADO music repository at the Bavarian State Library. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR), pp. 501–504 (2007)

  18. Dixon, S., Widmer, G.: MATCH: A music alignment tool chest. In: Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR) (2005)

  19. Dunn, J.W., Byrd, D., Notess, M., Riley, J., Scherle, R.: Variations2: Retrieving and using music in an academic setting. In: Pardo, B. (ed.): Special Issue: Music Information Retrieval, vol. 49, pp. 53–58. ACM, New York, NY, USA (2006). doi:10.1145/1145287.1145314

  20. European Union: EUROPEANA (2007). http://www.europeana.eu/portal/index.html

  21. Fremerey, C., Müller, M., Kurth, F., Clausen, M.: Automatic mapping of scanned sheet music to audio recordings. In: Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR), pp. 413–418. Philadelphia, USA (2008)

  22. Fremerey, C., Clausen, M., Ewert, S., Müller, M.: Sheet music-audio identification. In: Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR), pp. 645–650. Kobe, Japan (2009a)

  23. Fremerey, C., Müller, M., Clausen, M.: Towards bridging the gap between sheet music and audio. In: Selfridge-Field, E., Wiering, F., Wiggins, G.A. (eds.) Knowledge Representation for Intelligent Music Processing, no. 09051 in Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Germany, Dagstuhl, Germany (2009b). http://drops.dagstuhl.de/opus/volltexte/2009/1965

  24. Fremerey, C., Müller, M., Clausen, M.: Handling repeats and jumps in score-performance synchronization. In: Proceedings of the 11th International Conference on Music Information Retrieval (ISMIR). Utrecht, the Netherlands (2010)

  25. Good, M.: MusicXML: An internet-friendly format for sheet music. In: Proceedings XML Conference and Exposition (2001). http://www.idealliance.org/papers/xml2001/papers/html/03-04-05.html

  26. Google Inc.: Google Book Search (2007). http://books.google.com

  27. Goto, M.: A chorus-section detecting method for musical audio signals. In: Proceedings of the IEEE Internatinal Conference on Acoustics, Speech, and Signal Processing ICASSP, pp. 437–440 (2003)

  28. Gracenote: Music Search (2008). http://www.gracenote.com/

  29. Hankinson, A., Pugin, L., Fujinaga, I.: Interfaces for document representation in digital music libraries. In: Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR), pp. 39–44 (2009)

  30. Hu, N., Dannenberg, R., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. In: Proceedings of the 4th IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (2003)

  31. Huber D.M.: The MIDI Manual. Focal Press, Boston (1999)

    Google Scholar 

  32. IFLA Study Group: Functional requirements for bibliographic records: Final report. UBCIM Publications-New Series 19 (1998). http://www.ifla.org/VII/s13/frbr/frbr.htm

  33. Kahle, B.: Internet Archive (1996). http://www.archive.org/index.php

  34. Klapuri, A., Davy, M. (eds): Signal Processing Methods for Music Transcription. Springer, New York (2006)

    Google Scholar 

  35. Krajewski, E.: DE-PARCON softwaretechnologie (2008). http://www.de-parcon.de/

  36. Krottmaier, H., Kurth, F., Steenweg, T., Appelrath, H.J., Fellner, D.: PROBADO—a generic repository integration framework. In: Proceedings of the 11th European Conference on Research and Advanced Technology for Digital Libraries (ECDL) (2007)

  37. Kurth, F., Müller, M., Fremerey, C.: Audio Matching für symbolische Musikdaten. In: Fortschritte der Akustik, Tagungsband der DAGA (2007a).http://www.cs.uni-bonn.de/~meinard/publications/07_KuMuFr_DAGA_SymbAudioMatch.pdf

  38. Kurth, F., Müller, M., Fremerey, C., Chang, Y., Clausen, M.: Automated synchronization of scanned sheet music with audio recordings. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR), pp. 261–266 (2007b)

  39. Kurth F., Müller M.: Efficient index-based audio matching. IEEE Trans. Audio Speech Lang. Process. 16(2), 382–395 (2008)

    Article  Google Scholar 

  40. Landone, C., J., H., Reiss, J.: Enabling access to sound archives through integration, enrichment and retrieval: the EASAIER project. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR), pp. 159–160 (2007)

  41. Ludovico L.A.: IEEE 1599: a multi-layer approach to music description. J. Multimed. 4(1), 9–14 (2009)

    Google Scholar 

  42. Maddage, N.C., Xu, C., Kankanhalli, M.S., Shao, X.: Content-based music structure analysis with applications to music semantics understanding. In: Proceedings of the ACM Multimedia, pp. 112–119. New York, NY, USA (2004). doi:10.1145/1027527.1027549

  43. Müller M.: Information Retrieval for Music and Motion. Springer, New York (2007)

    Book  Google Scholar 

  44. Müller, M., Appelt, D.: Path-constrained partial music synchronization. In: Proceedings of the 34th International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 1, pp. 65–68. Las Vegas, Nevada, USA (2008)

  45. Müller, M., Clausen, M.: Transposition-invariant self-similarity matrices. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007), pp. 47–50 (2007)

  46. Müller M., Kurth F.: Towards structural analysis of audio recordings in the presence of musical variations. EURASIP J. Appl. Signal Process. 2007(89686), 18 (2007)

    Google Scholar 

  47. Müller, M., Kurth, F., Röder, T.: Towards an efficient algorithm for automatic score-to-audio synchronization. In: Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR), pp. 365–372. Barcelona, Spain (2004)

  48. Müller, M., Kurth, F., Damm, D., Fremerey, C., Clausen, M.: Lyrics-based audio retrieval and multimodal navigation in music collections. In: Proceedings of the 11th European Conference on Research and Advanced Technology for Digital Libraries (ECDL) (2007)

  49. Orio, N.: Alignment of performances with scores aimed at content-based music access and retrieval. In: Proceedings of the 6th European Conference on Research and Advanced Technology for Digital Libraries (ECDL), pp. 479–492. Rome, Italy (2002)

  50. Orio, N., Lemouton, S., Schwarz, D.: Score following: State of the art and new developments. In: Proceedings of the Conference of New Interfaces for Musical Expression (NIME), pp. 36–41. Montreal, CA (2003)

  51. Pardo, B.: Introduction. In: Pardo, B. (ed.): Special Issue: Music Information Retrieval, vol. 49, pp. 28–31. ACM, New York, NY, USA (2006). doi:10.1145/1145287.1145309

  52. Peeters, G., Burthe, A.L., Rodet, X.: Toward automatic music audio summary generation from signal analysis. In: Proceedings of the 3th International Conference on Music Information Retrieval (ISMIR) (2002)

  53. Pickens, J., Bello, J.P., Monti, G., Crawford, T., Dovey, M., Sandler, M.: Polyphonic score retrieval using polyphonic audio queries: a harmonic modeling approach. In: Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR), pp. 140–149. Paris, France (2002)

  54. Pinto A.: Multi-model music content description and retrieval using IEEE 1599 XML standard. J. Multimed. 4(1), 30–39 (2009)

    Google Scholar 

  55. Raphael, C.: A hybrid graphical model for aligning polyphonic audio with musical scores. In: Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR) (2004)

  56. Rauber, A., Frühwirth, M.: Automatically analyzing and organizing music archives. In: Proceedings of the 5th European Conference on Research and Advanced Technology for Digital Libraries (ECDL), Springer Lecture Notes in Computer Science. Springer, Darmstadt, Germany (2001). http://www.ifs.tuwien.ac.at/ifs/research/publications.html

  57. Selfridge-Field, E. (eds): Beyond MIDI: The Handbook of Musical Codes. MIT Press, Cambridge (1997)

    Google Scholar 

  58. Soulez, F., Rodet, X., Schwarz, D.: Improving polyphonic and poly-instrumental music to score alignment. In: Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR) (2003)

  59. Suyoto I.S.H., Uitdenbogerd A.L., Scholer F.: Searching musical audio using symbolic queries. IEEE Trans. Audio Speech Lang. Process. 16(2), 372–381 (2008). doi:10.1109/TASL.2007.911644

    Article  Google Scholar 

  60. Turetsky, R.J., Ellis, D.P.: Force-aligning MIDI syntheses for polyphonic music transcription generation. In: Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR) (2003a)

  61. Turetsky, R.J., Ellis, D.P.W.: Ground-truth transcriptions of real music from force-aligned MIDI syntheses. In: Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR) (2003b)

  62. Typke, R., Wiering, F., Veltkamp, R.C.: A survey of music information retrieval systems. In: Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR), pp. 153–160 (2005)

  63. Union der deutschen Akademien der Wissenschaften: Neue Mozart Ausgabe (2007). http://www.nma.at/

  64. United States: World Digital Library (2009). http://www.wdl.org/en/

  65. University of Chicago Library: Chopin Early Edition (2004). http://chopin.lib.uchicago.edu/

  66. University of Rochester Libraries: UR research—Sibley Music Library (2009). https://urresearch.rochester.edu/home.action

  67. W3C: Web Services. http://www.w3.org/2002/ws/

  68. Wang, A.L.C.: An industrial-strength audio search algorithm (2003). http://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf

  69. Wang, Y., Kan, M.Y., Nwe, T.L., Shenoy, A., Yin, J.: LyricAlly: automatic synchronization of acoustic musical signals and textual lyrics. In: Proceedings of the 12th annual ACM International Conference on Multimedia, pp. 212–219. ACM Press, New York, NY, USA (2004). http://doi.acm.org/10.1145/1027527.1027576

  70. Wiener Wissenschafts-, Forschungs- und Technologiefonds: Schubert-Autographe. http://www.schubert-online.at/

  71. Witten I.H., Moffat A., Bell T.C.: Managing Gigabytes. 2nd edn. Van Nostrand Reinhold, New York (1999)

    Google Scholar 

  72. Witten, I.H., Mcnab, R.J., Boddie, S.J., Bainbridge, D.: Greenstone: A comprehensive open-source digital library software system. In: Proceedings of the 5th ACM International Conference on Digital Libraries (2000). http://citeseer.ist.psu.edu/witten99greenstone.html

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science III, University of Bonn, Römerstr. 164, 53117, Bonn, Germany

    David Damm, Christian Fremerey, Verena Thomas & Michael Clausen

  2. Fraunhofer Institute for Communication, Information Processing and Ergonomics (FKIE), Neuenahrer Str. 20, 53343, Wachtberg, Germany

    Frank Kurth

  3. Saarland University and Max-Planck-Institut für Informatik, Campus E1 4, 66123, Saarbrücken, Germany

    Meinard Müller

Authors
  1. David Damm
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Fremerey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Verena Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Clausen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Frank Kurth
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Meinard Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Damm.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Damm, D., Fremerey, C., Thomas, V. et al. A digital library framework for heterogeneous music collections: from document acquisition to cross-modal interaction. Int J Digit Libr 12, 53–71 (2012). https://doi.org/10.1007/s00799-012-0087-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00799-012-0087-y

Keywords

Search

Navigation