Data-driven spectral basis functions for automatic speech recognition

Abstract

Feature extraction plays a major role in any form of pattern recognition. Current feature extraction methods used for automatic speech recognition (ASR) and speaker verification rely mainly on properties of speech production (modeled by all-pole filters) and perception (critical-band integration simulated by Mel/Bark filter bank). We propose to use stochastic methods for designing feature extraction methods which are trained to alleviate the unwanted variability present in speech signals. In this paper we show that such data-driven methods provide significant advantages over the conventional methods both in terms of performance of ASR and in providing understanding about the nature of speech signal. The first part of the paper investigates the suitability of the cepstral features obtained by applying discrete cosine transform on logarithmic critical-band power spectra. An alternate set of basis functions were designed by linear discriminant analysis (LDA) of logarithmic critical-band power spectra. Discriminant features extracted by these alternate basis functions are shown to outperform the cepstral features in ASR experiments. The second part of the paper discusses the relevance of non-uniform frequency resolution used by current speech analysis methods like Mel frequency analysis and perceptual linear predictive analysis. It is shown that LDA of the short-time Fourier spectrum of speech yields spectral basis functions which provide comparatively lower resolution to the high-frequency region of spectrum. This is consistent with critical-band resolution and is shown to be caused by the spectral properties of vowel sounds.

Introduction

A typical large vocabulary automatic speech recognition (ASR) system consists of three main components: feature extraction, pattern classification, and language modeling. Feature extraction attempts to reduce the data rate of raw speech data by alleviating irrelevant variability such as speaker characteristics or environmental noise. The pattern classification module uses statistical models, mainly Hidden Markov models (HMMs), to determine the most likely sequence of sounds that could have generated the observed sequence of feature vectors. The language model steers the classification module to generate word sequences that are more probable.

While many of the early ASR systems used stochastic methods for pattern classification (Vintsyuk, 1968; Jelinek, 1975; Sakoe and Chiba, 1978), some early ASR systems were inspired by the advances in artificial intelligence (Lesser et al., 1975). These systems relied on sets of rules for pattern classification and language modeling. These rules were often prescribed by experts in reading spectrograms and hand crafted for specific recognition problems at hand (Reddy, 1976; Zue, 1990; Mercier et al., 1990). They worked reasonably well for small tasks under controlled environments. The performance of such systems was found to be fragile (Klatt, 1977).

The past two decades have witnessed a significant increase in stochastic approaches in both the pattern classification and the language modeling modules. These stochastic approaches brought the rich mathematical basis that was available in the classical pattern recognition literature to ASR. In the current ASR systems, the pattern classification module uses HMMs (Jelinek, 1997; Rabiner and Juang, 1993) and artificial neural networks (Morgan and Bourlard, 1995) while the language models are in the form of N-grams which are trained from a large text corpus (Katz, 1987). Stochastic techniques typically use only minimal a priori assumptions about the nature of the problem. Such techniques estimate the parameters of models directly from the data. Replacing the hardwired prior knowledge by the knowledge derived from the data turned out to be one of the more significant advances in ASR research.

Although statistical methods are dominant in pattern classification and language modeling, the methods used to derive features for speech recognition are still based on knowledge of the human auditory system. Mainstream systems simulate auditory filters by applying a weighting function on the short-time Fourier spectrum (Mermelstein, 1976; Hermansky, 1990). The output of these weighting functions is then passed through a logarithmic non-linearity and finally projected into the cosine basis functions (discrete cosine transform, DCT).

In this paper we first investigate the optimality of using DCT to extract features from the outputs of a non-uniform filter bank. In Section 2 it will be shown that DCT can at best be equivalent to a principal component transform which preserves the maximum amount of variance while reducing the dimensionality of the feature space. In Section 3 we review the theory behind linear discriminant analysis (LDA) and show in Section 4 that basis functions derived using LDA (designed to preserve maximum phonetic class separability) outperform the DCT (Hermansky and Malayath, 1998). The need for critical-band analysis itself (which implies non-uniform resolution) is investigated in Section 5. Use of critical-band analysis (or Mel filter-bank analysis) is motivated by properties of hearing. This analysis provides higher resolution to the low-frequency region of speech spectrum. We investigate the optimality of such an analysis from the point of view of pattern recognition. Through such an analysis we show that the Mel/Bark-like frequency resolution automatically results from discriminant analysis of short-time Fourier spectrum of speech. We go on to show that this non-uniform resolution can be traced to the physiology of speech production mechanism. There has been work done earlier in relating the Mel/Bark scale to the properties of speech signal and speech production mechanism. Mel-like warping has been shown to be optimal for normalizing speaker variability (Umesh et al., 1997; Kamm et al., 1997). To our knowledge, the Mel/Bark scale was first shown to result from LDA by Hermansky and Malayath (1998). Hunt argued that the advantage of non-uniform spacing of channels is due to the acoustical properties of voiced sounds (Hunt, 1999). A detailed analysis of the relation between speech production and the optimality of non-uniform frequency resolution can be found in (Malayath, 2000).