DISTBIC: A speaker-based segmentation for audio data indexing

Abstract

In this paper, we address the problem of speaker-based segmentation, which is the first necessary step for several indexing tasks. It aims to extract homogeneous segments containing the longest possible utterances produced by a single speaker. In our context, no assumption is made about prior knowledge of the speaker or speech signal characteristics (neither speaker model, nor speech model). However, we assume that people do not speak simultaneously and that we have no real-time constraints. We review existing techniques and propose a new segmentation method, which combines two different segmentation techniques. This method, called DISTBIC, is organized into two passes: first the most likely speaker turns are detected, and then they are validated or discarded. The advantage of our algorithm is its efficiency in detecting speaker turns even close to one another (i.e., separated by a few seconds).

Zusammenfassung

Dieser Artikel beschreibt Sprecher basierte Segmentierung, den ersten Schritt beim Indexieren von Sprechern. Das Ziel besteht darin, möglichst lange homogene Segmente zu extrahieren, die Laute eines einzelnen Sprechers enthalten. Wir legen zugrunde, daß keinerlei Sprachcharakteristik des Sprechers bekannt ist (weder Sprechermodel noch Sprachmodel). Außerdem wird die Annahme gemacht, daß immer nur ein Sprecher zur Zeit spricht und daß keine Echtzeitanforderungen vorhanden sind. Wir stellen existierende Segmentierungtechniken vor und schlagen eine neue Methode vor, welche zwei gebräuchliche Methoden kombiniert. Unsere Methode (DISTBIC) ist in zwei Phasen aufgeteilt: erst werden die wahrscheinlichsten Sprecherwechsel gefunden, die dann entweder validiert oder verworfen werden. Der Vorteil unseres Algorithmuses liegt in seiner Effizienz Sprecherwechsel aufzufinden, besonders wenn sie sehr nahe beieinander liegen (d.h. Abstände von wenigen Sekunden).

Résumé

Dans cet article, nous nous intéressons au problème de la segmentation en locuteurs, étape préliminaire nécessaire à plusieurs tâches d'indexation. Le but de la segmentation en locuteurs est d'extraire des segments homogènes ne contenant les paroles que d'un seul locuteur et aussi longs que possible. Dans notre contexte, nous faisons l'hypothèse qu'aucune connaissance a priori des locuteurs ou des caractéristiques du signal n'est à notre disposition (pas de modèle de locuteur, pas de modèle de parole). Nous supposons néanmoins que les personnes ne parlent pas simultanément et que nous n'avons pas de contrainte de temps réel. Nous présentons les techniques de segmentation existantes et nous proposons une nouvelle méthode qui combine les avantages de deux techniques de segmentation. Cette nouvelle méthode de segmentation, appelée DISTBIC, s'opère en deux passes: les changements de locuteurs les plus probables sont tout d'abord détectés et ils sont ensuite validés ou annulés au cours de la deuxième passe. L'avantage de notre algorithme est son efficacité à détecter des changements de locuteurs proches les uns des autres (i.e. espacés de quelques secondes).

Introduction

With the ever increasing number of TV channels and broadcasting radio stations and thanks to the currently available huge storage means, many hours of TV and radio broadcasts are collected every year by information national heritage institutions, like the Institut National de l'Audiovisuel (INA) in France or the BBC archives in the UK. For example, INA possesses 45 years of TV archives consisting of 300,000 hours of national TV programs and 60 years of radio archives consisting of 400,000 hours of radio programs. Moreover, due to systematic digitalization of information, multimedia databases are on a rocketing increase.

Besides the storage and architecture problems underlying the design of such databases, another crucial problem is information retrieval: how to formulate a query in a convenient way and how to efficiently and quickly find the searched information whatever it could be: text, drawings, image, video, audio, music or speech. Pre-indexing is necessary to facilitate and speed-up any kind of query.

Clearly, access to audio documents is much more difficult than access to text; although text retrieval must cope with some variability of spelling by proposing different approximated solutions to the user, it is still easier to detect a name or a string of words in a text than to recognize a speaker or to spot a word within an audio recording, or to recognize a spoken sentence in a large lexicon. Also, listening to an audio recording takes much more time than reading a text. Consequently, it is essential to be able to directly access the significant segments rather than listening to the whole audio recording to retrieve pertinent information.

Audio document indexing associates with each audio document a file describing its structure in terms of retrieval keys. Phoneme strings can be keys for retrieval of a word or a sentence in a speech file (word- and sentence-spotting). Topic spotting plays an essential role in document filtering and understanding. Another key could be speaker identity. The presence of a given speaker in a conversation could be detected if this speaker's voice characteristics have been a priori enrolled. Automatic analysis of conversations recordings requires segmentation into segments containing only one speaker and segment clustering into one-speaker sets.

In this paper, we mainly address the specific problem of audio database segmentation with respect to speakers which is an essential initial step towards full indexing. To stay close to the application, no assumption is made about prior knowledge of the speaker or speech signal characteristics. However, we assume that people do not speak simultaneously. Additionally, since the construction of an index file is an off-line process, we have no real-time constraints. The problem of speaker-based segmentation and indexing is stated in Section 2. Possible application fields are described in Section 2.2. The hypotheses made for this work are discussed: they place our work in the perspective of approaches followed by other authors. Section 3 presents a brief description of the pioneering indexing tool of BBN for application in air traffic control. Section 4 deals with the segmentation operation which is central in this paper and makes a short review of different proposed techniques including the inspiring technique used by Chen and Gopalakrishnan (1998) at IBM. At this point, a new original technique DISTBIC is proposed using a two-pass approach. Since no prior knowledge about speakers is used, our solution turns out to be close to a general change detection algorithm. However, the application to sequences of feature vectors extracted from the speech waveform and containing speaker information puts specific tunings forward and keeps the general principle in the background. Different criteria are presented as well as the complete algorithm for speaker turn detection. The role of computational effort required is not crucial while the completeness of the segmentation, whatever the speaker intervention lengths are, is essential. Improving this completeness is the aim of the proposed algorithm. The results of DISTBIC are reported in Section 6. We conclude and describe perspectives towards the complete realization of an indexing tool in Section 7.