Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems
Abstract
The NOISEX-92 experiment and database is described and discussed. NOISEX-92 specifies a carefully controlled experiment on artificially noisy speech data, examining performance for a limited digit recognition task but with a relatively wide range of noises and signal-to-noise ratios. Example recognition results are given.
Zusammenfassung
Es werden die Datenbank und die Tests NOISEX-92 beschrieben und kommentiert. NOISEX-92 spezifiziert Tests über künstliche Lärmdaten, wobei die Leistungen im Rahmen der Erjennung von Zahlen und ein relativ breiter Rauschabstand bewertet werden. Es werden Beispiele der Ergebnisse für Erkennungen beschrieben.
Résumé
La base de données et les tests NOISEX-92 sont décrits et commentés. NOISEX-92 spécifie des tests sur des données bruitées artificiellement en évaluant les performances dans le cadre de la reconnaissance de chiffres et une gamme relativement large de rapports signal sur bruit. Des examples de scores de reconnaissances sont présentés.
References (6)
- H.J.M. Steeneken et al.
Assessment for automatic speech recognition: I. Comparison of assessment methods
Speech Communication
(1993) - H.J.M. Steeneken et al.
Comparison of some methods for measuring speech levels
- H.J.M. Steeneken et al.
Description of the RSG.10 Noise data-base
Cited by (1933)
A speech enhancement model based on noise component decomposition: Inspired by human cognitive behavior
2024, Applied AcousticsIn multimedia intelligent systems, speech enhancement is commonly employed to improve the quality of speech signals, making them clearer and more natural. Current deep learning-based speech enhancement models typically treat noise as a unified entity and aim to separate it from the target speech. In this paper, inspired by the cognitive behavior of the human brain when observing noisy speech spectrograms, we decompose the spectral energy of noise into regular and random components. We propose an auxiliary-model-based speech enhancement framework that better suppresses noise components closely resembling speech features. Firstly, we introduce a voiceprint segmentation network (VSnet) that partitions noisy speech into voiceprint and non-voiceprint regions. Subsequently, we present a noise reconstruction network (NRnet) that utilizes noise information from non-voiceprint regions to reconstruct and suppress the regular noise components within the voiceprint region. Finally, we construct a combination of a model dedicated to suppressing random components (RANnet) and a speech enhancement model (SEnet), and train them synchronously. By sharing encoder parameters, SEnet is compelled to reduce the extraction of regular noise features from the original noisy speech, contributing to improving speech quality generated through the decoder. Experimental results on public Voickbank-DEMAND and DNS-challenge 2020 datasets demonstrate that our approach achieves state-of-the-art performance.
Noise-robust voice conversion using adversarial training with multi-feature decoupling
2024, Engineering Applications of Artificial IntelligenceMost existing voice conversion methods focus primarily on separating speech content from speaker information while overlooking the decoupling of pitch information. Additionally, the quality of converted speech significantly degrades when the speech of the target speaker is contaminated by noises. To address these issues, this paper proposes a noise-robust voice conversion model with multi-feature decoupling based on adversarial training. The proposed framework utilizes three distinct encoders to encode speech content, speaker identity, and pitch information independently, which aims to enhance the performance of decoupling by minimizing their mutual information and reduce the correlations between feature vectors. Moreover, a gradient reversal layer and a noise decoupling discriminator are incorporated into the framework, which extracts noise-resistant speaker representations and content representations through adversarial training to facilitate the synthesis of high-quality speech. In order to optimize the learning process, a training strategy is developed which involves alternating between clean and noisy data during the training of the encoder. This strategy effectively guides and expedites the convergence of the model. Experimental results demonstrate that compared to the state-of-the-art baselines of noise-robust voice conversion, the proposed model achieves improvements around 0.31 and 0.39 in terms of speech naturalness and speaker similarity evaluation metrics, respectively.
Single-layer folded RNN for time series prediction and classification under a non-Von Neumann architecture
2024, Digital Signal Processing: A Review JournalA delay dynamical system can fold a feedforward neural network into one nonlinear neuron and multiple delay loops under the non-Von Neumann structure, greatly decreasing the hardware requirements. In this paper, we transform the folded-in-time DNN (Fit-DNN) into a folded-in-time RNN (Fit-RNN) and derive the backpropagation algorithm for it. The performance of the folded reservoir computing (DRC), Fit-DNN, and Fit-RNN is compared on time-series prediction and classification tasks, respectively. The impact of virtual node separation on the performance of Fit-RNN is analyzed. The limits of Fit-RNN's capabilities in conducting effective time-series predictions were determined based on the NARMA-X series tasks. We calculate the 5-order information processing capacity (IPC) of DRC and Fit-RNN on NARMA10. The results indicate that Fit-RNN has almost the necessary information transformation ability for the task. In the recognition tasks conducted separately on spoken digits and speakers, we investigate the impact of the number and separation of virtual nodes on the recognition capability of the three folded networks. The results demonstrate that Fit-RNN shows more promise in handling long sequences and large-scale recognition tasks. The recognition accuracy is further enhanced by increasing the number of virtual nodes and node separation. Furthermore, the introduction of two common types of noise in the speaker recognition task environment further highlights the potential of Fit-RNN in practical applications. Furthermore, two common types of noise are introduced into the speaker recognition task, further highlighting the potential of Fit-RNN in practical applications.
Cortical tracking of speech is vital for speech segmentation and is linked to speech intelligibility. However, there is no clear consensus as to whether reduced intelligibility leads to a decrease or an increase in cortical speech tracking, warranting further investigation of the factors influencing this relationship. One such factor is listening effort, defined as the cognitive resources necessary for speech comprehension, and reported to have a strong negative correlation with speech intelligibility. Yet, no studies have examined the relationship between speech intelligibility, listening effort, and cortical tracking of speech. The aim of the present study was thus to examine these factors in quiet and distinct adverse listening conditions. Forty-nine normal hearing adults listened to sentences produced casually, presented in quiet and two adverse listening conditions: cafeteria noise and reverberant speech. Electrophysiological responses were registered with electroencephalogram, and listening effort was estimated subjectively using self-reported scores and objectively using pupillometry. Results indicated varying impacts of adverse conditions on intelligibility, listening effort, and cortical tracking of speech, depending on the preservation of the speech temporal envelope. The more distorted envelope in the reverberant condition led to higher listening effort, as reflected in higher subjective scores, increased pupil diameter, and stronger cortical tracking of speech in the delta band. These findings suggest that using measures of listening effort in addition to those of intelligibility is useful for interpreting cortical tracking of speech results. Moreover, reading and phonological skills of participants were positively correlated with listening effort in the cafeteria condition, suggesting a special role of expert language skills in processing speech in this noisy condition. Implications for future research and theories linking atypical cortical tracking of speech and reading disorders are further discussed.
A ChannelWise weighting technique of slice-based Temporal Convolutional Network for noisy speech enhancement
2024, Computer Speech and LanguageIn recent years, Temporal Convolutional Networks (TCNs) have driven significant progress in single-channel noisy speech enhancement. However, TCN-based systems still face certain challenges, such as limited utilization of network channel depth for handling long-range dependencies and issues with weight sharing. To address these challenges, this paper proposes a novel channel-wise weighting scheme, specifically designed for the sliced TCN framework. The proposed scheme involves the element-wise multiplication of shifting weight techniques for each channel of the TCN slice. Utilizing a cyclically shifted approach, these weights capture information from neighboring channels, uncovering the dependencies between adjacent channels. By combining the channel-wise weighted TCN output and subsequently estimating a masking function, the proposed method effectively suppresses noise components, leading to enhanced speech quality. To train and evaluate our proposed method, we utilize speech datasets that consist of various noise types at different levels. To optimize the performance of the proposed end-to-end enhancement system, we adopt the Scale-Invariant Signal-to-Noise Ratio (SI-SNR) objective function. Experimental results demonstrate the effectiveness of our proposed TCN channel-wise weighting method, with a significant average improvement of approximately 9.8% in SI-SNR for the unseen noise dataset. This improvement was observed at an SNR of 3 dB for both non-channel-wise weighting schemes and the proposed channel-wise weighting schemes within the Multi-slicing TCNs framework. The main advantage of the proposed approach is its ability to address the challenges of uneven and biased output from TCN slices, particularly when dealing with highly non-stationary, noisy speech signals infused with speech-like noise. This leads to more robust performance in various real-world applications.
A review on speech emotion recognition: A survey, recent advances, challenges, and the influence of noise
2024, NeurocomputingAffective Computing systems can detect the emotional state and mindset of an individual. Speech Emotion Recognition (SER) is a unimodal affect computing system based on emotional speech data. It is an active area of research in pattern recognition, computer vision, and deep learning. There is a great deal of literature on SER, but only a few of these works consider how SER performs under noisy conditions. A few surveys exist to review SER, but they either need to cover all aspects of SER in noisy environments or discuss the details thoroughly. In recent years, researchers have had a growing interest in using SER in real-world conditions and have seen improvements in recognition rate. This review compiles the methods and approaches used in noisy SER in the literature up to the mid of 2023. It covers topics such as noisy SER methods, datasets used for SER under noisy conditions, noise used, and toolkits used for noisy SER recognition. Additionally, it focuses on classifiers, features used, and limitations of existing research in noisy SER systems. The review also seeks to answer "Does noise affects performance?" to which the answer is a resounding yes, as demonstrated by the results obtained from this survey.