Automatic identification of audio recordings based on statistical modeling

Abstract

This paper describes a methodology for the automatic identification of audio recordings of ethnic music. The identification is based on an application of hidden Markov models (HMMs), which are automatically built from a representation of the music pieces to be identified. States of the HMMs are labeled with music events, and the transition and observation probabilities are directly computed from the information on the music piece. The recordings are modeled by a set of acoustic features that are computed according with the characteristics of the music events. Three alternative approaches, based on typical applications of HMMs, are proposed to perform the identification. Tests carried out on collections of recordings showed that the methodology can achieve good results, and the identification rate is high enough to suggest applications for automatic retrieval of metadata and for the identification of alternative recordings of the same piece.

Introduction

Digital collections, usually stored and organized in digital libraries, are becoming an increasingly important tool for the preservation and dissemination of music cultural heritage. Music can benefit from the application of digital technologies even more than other art forms, because it can be shared by users with different background and education, easily crossing language barriers. This is particularly true for ethnic music, which can be enjoyed by users with different culture, traditions, and living in different geographical areas. For a digital library of ethnic music, the goal “anytime–anywhere” is meant in its widest sense. Yet, the interest towards music digital collections should be paired by the availability of tools for an effective access to digital content. Textual descriptors such as title, author, and performers are all valuable metadata that can allow a user to retrieve potentially relevant music items. Moreover, additional information about time and key signatures, instrumentation, and geographical location, may provide alternative ways to access music content.

In the case of ethnic music, the access to music documents becomes more challenging, because typical metadata may not be good descriptors of music content. The most effective fields in a music search—author and title according to [1]—may not be good descriptors for this repertoire. To this end, tools for content-based search should be provided to the user, aiming at the exploration of the digital collection, for instance to retrieve alternative recordings of a given piece also when textual metadata are different or not available. Another important issue on the development of a music digital collection regards how music content is cataloged during the acquisition of the material. Classification and labeling should be manually carried out by trained personnel, with a considerable increment in time and costs. Also in this case, automatic tools that allow the identification of a recording as an instance of a given song can be used to automatically add pertinent metadata.

This paper reports a methodology for the automatic identification of unknown recordings. The approach is based on hidden Markov models (HMMs), which model the unobservable process underlying the production of a music performance given a statistical representation of a music piece. The presented approach builds on earlier work on audio to score alignment [2], [3], which has been extended to the particular task of exploring ethnic music digital collections. One of the goals of the proposed paper is the exploration on whether HMM-based identification can be carried out also when partial information—e.g., melodic content—is available to describe a song. To this end, some of the common characteristics of many repertoires of ethnic music make the proposed approach particularly suitable.

First of all, melody plays a fundamental role in most ethnic repertories, as opposed to functional harmony used in tonal Western music, yet the same song can be performed in a countless number of variants, which depend on the oral tradition by which it is transmitted, on the available instrumentation, on the geographical areas and thus on the cultures that have been influenced by it. Given the prominent role of oral tradition, for many songs the most common metadata such as author and title may not be significant—the author being unknown and the title being present in dozens of variants—making music retrieval a particularly difficult task. The fact that a given piece can be sung a cappella, played by a single instrument, played or sung with accompaniment, makes its identification challenging because only partial information about the acoustic features is available. All these considerations suggest the use of a statistical framework to model ethnic music recordings. The framework can be applied to identify unknown recordings, mine a music collection to highlight the presence of different recordings—for instance due to geographical variants—of the same song.

The paper is organized as follows. Section 2 introduces a number of approaches that are related to the presented methodology. The methodology for music modeling is described in Section 3, while three approaches to identification are presented in Section 4. The results of the experimental evaluation are given in Section 5, followed by some concluding considerations in Section 6.