On the relationship between Early-to-Late Ratio of Room Impulse Responses and ASR performance in reverberant environments

Highlights

• Study on the correlation between ASR accuracy and reverberant acoustic conditions.

• Experiments involve a large variety of simulated and measured room impulse responses.

• The best duration of early arrivals is determined experimentally.

• The approach is applied to data contamination for acoustic model training and model selection.

• A large vocabulary recognition task (WSJ) is considered using both GMM and DNN.

Abstract

This work presents an experimental analysis of distant-talking speech recognition in a variety of reverberant conditions, correlating ASR performance to a compact representation of the propagation channel (i.e., the room impulse response).

It is well known that reverberation and background noise degrade speech recognition performance, but few studies have investigated the relation between room impulse responses and recognition rates in a comprehensive manner. In particular, we show how the ASR accuracy is related to features derived from the structure of the early arrivals and the reverberation tail. A representation based on the combination of few parameters is hence proposed, analysing the impact of reverberation on different speech recognition tasks. Possible applications of the derived measure are in data contamination for acoustic modeling where this feature can be employed either to select the most suitable model for a given acoustic condition or to define the subset of room impulse responses to be used for the creation of partially matched reverberant models. Recognition results using different back-end solutions (GMM, DNN) on data generated with the image method and with real impulse responses validate the effectiveness of the approach.

Introduction

Automatic speech recognition (ASR) can be considered a reliable technology only in controlled environments where the quality of the acquired voice signal is not critically degraded by environmental factors as background noise or reverberation. Close-talking microphones are often required to guarantee such quality but in specific scenarios these microphones are inconvenient or intrusive. As a result, distant speech recognition is gaining major attention and a number of studies have been addressing the need of effective speech acquisition and processing (Wölfel and McDonough, 2009). As primary way, microphone arrays allow the implementation of selective spatial acquisition or other speech enhancement techniques (Brandstein and Ward, 2001), like beamforming (Benesty et al., 2008), dereverberation and denoising (Naylor and Gaubitch, 2010). However, compact microphone arrays have a restricted spatial view and are therefore not suitable for wide enclosures where the source may be in an unfavorable position (e.g., not frontal to the microphones, without line of sight). As an alternative, networks of distributed microphones could be employed, guaranteeing a uniform acoustic coverage of the monitored area, independently of the source position and orientation. The latter scenario and the related challenges have been investigated under the EU project DIRHA, where a vocal system for the control of home devices is targeted. In such configurations the adoption of array-processing techniques is often neither possible nor effective (Kumatani et al., 2011) and alternative approaches can be successfully applied, as for example channel selection (Wolf and Nadeu, 2014) or source separation (Vincent et al., 2013, Delcroix et al., 2013).

Although several studies investigated the decreasing trend of ASR performance as a function of the actual noise (i.e., the SNR), few studies have addressed the relation between the acoustic propagation and recognition rates (Nishiura et al., 2007, Fukumori et al., 2013). Indeed, in a distributed microphone setup, various spatial factors have an impact on the signal quality: the distance and the orientation of the pairs source-microphone, the consequent different SNR, the acoustic propagation. Hence, it is of interest to correlate ASR performance with some purely acoustic measurements. In this direction an estimation of the Signal-to-Noise Ratio (SNR) in multiple sub-bands has been introduced in (Takeda et al., 2000). In (Hermansky et al., 2013), the authors proposed a temporal-domain method for predicting recognition performance in unseen noisy environments. This estimate can be usefully exploited during setup to increase the robustness of the resulting system, for example selecting or training a suitable acoustic model. Authors in (Petrick et al., 2007) studied the harming parts of room impulse responses, discussing the contribution of early and late reflections to ASR performance, while in (Sehr and Kellermann, 2010) the inter-frame correlation of reverberant feature vectors is analyzed. In a related work, the adjustments of dereverberation algorithms to ASR systems are evaluated (Sehr et al., 2010). More recently the reverberation problem has been addressed from different perspectives (Krueger and Haeb-Umbach, 2010, Sadjadi and et al., 2012, Gomez et al., 2013, Wang et al., 2012), showing the need for effective solutions to cope with the related masking effects.

This work presents an experimental analysis of distant-talking speech recognition in reverberant conditions, correlating the recognition performance to the acoustic characteristics of the propagation channel, namely the structure of the Room Impulse Response (RIR). It extends our previous work (Brutti and Matassoni, 2014), providing a more exhaustive experimental analysis and considering a large number of acoustic conditions (including two databases of real measured RIRs). We show that, given a certain recognition task (e.g., WSJ), the recognition accuracy varies according to the actual reverberant condition and is directly related to the Early-to-Late Reverberation Ratio (ELR) of the corresponding RIR, similarly to the SNR in case of background noise. In particular, we compare three different back-end solutions, based on Gaussian Mixture Model (GMM), Feature space Maximum Likelihood Linear Regression (fMLLR) and Deep Neural Networks (DNN), to further validate our hypothesis, independently of the recognition approach.

Basically, given a variety of acoustic conditions, typical of a domestic scenario (different room layouts, source and microphone positions, orientations and directivity patterns), ASR performance in reverberant conditions can be predicted using few parameters able to characterize the acoustic environment. As a result, a suitable data contamination strategy (Matassoni et al., 2002) for acoustic modeling can be derived, associating reverberant conditions with similar ELR to the same acoustic model, independently of the actual room characteristics. Indeed, given a set of pre-trained acoustic models associated to a variety of reverberant conditions, it is possible to select the best model as the model whose ELR closely matches the ELR of the test channel. A further application is in multi-condition training: the addressed ELR metric can be used to efficiently define a limited set of acoustic conditions that are sufficient to derive a robust multi-condition model.

Note that the estimation of the ELR, or similar metrics, directly from the recorded audio signals is beyond the scope of this experimental analysis. Recently, solutions employing GMM (Matassoni and Brutti, 2014) or DNN (Parada et al., 2014) classifiers have been presented. Several methods addressing the estimation of similar metrics are available in the literature (Naylor et al., 2010, Jeub et al., 2011, Falk and Chan, 2010, Georganti et al., 2014). Finally, Blind System Identification (BSI) algorithms (Kowalczyk et al., 2013, Huang and Benesty, 2003) could be employed to obtain an estimation of the propagation channel from which the ELR metric can be obtained.

The paper is organized as follows. Section 2 introduces the problem of ASR in reverberant conditions while Section 3 presents the proposed characterization of the impulse responses based on a low-rank representation. The experimental framework is introduced in Section 5, describing the data and the recognition tasks. Results on simulated data are discussed in Section 6 while results obtained on real measured RIRs are reported in Sections 7 Results on AIR RIRs, 8 Results on DIRHA RIRs. Finally, Section 9 draws some conclusions and introduces possible directions for future investigations.

Metrics similar to the ELR, like the definition and clarity, have been already introduced in (Kuttruff, 1991) without any reference to ASR performance or acoustic modeling in general. In (Nishiura et al., 2007, Fukumori et al., 2013), the correlation between these two metrics and the ASR performance is investigated with interesting results. Recently, the definition was adopted in (Sehr et al., 2010) in relation with the performance of de-reverberation algorithms. Similarly, in (Couvreur and Couvreur, 2000, Wolf and Nadeu, 2014), equivalent metrics are adopted to select the most promising microphones for ASR. However, their correlation with the recognition performance is not investigated. Recently, some works have targeted the model selection based on the similarity between the definition of the test and training channel (Parada et al., 2014, Xiong et al., 2014). In this paper, we propose a novel representation that is more effective than other descriptors commonly used in the literature.