Data-Driven Jacobian Adaptation in a Multi-model Structure for Noisy Speech Recognition

Abstract

We propose a data-driven approach for the Jacobian adaptation (JA) to make it more robust against the noisy environments in speech recognition. The reference hidden Markov model (HMM) in the JA is trained directly with the noisy speech for improved acoustic modeling instead of using the model composition methods like the parallel model combination (PMC). This is made possible by estimating the Jacobian matrices and other statistical information for the adaptation using the Baum-Welch algorithm during the training. The adaptation algorithm has shown to give improved robustness especially when used in a multi-model structure. From the speech recognition experiments based on HMMs, we could find the proposed adaptation method gives better recognition results compared with conventional HMM parameter compensation methods and the multi-model approach could be a viable solution in the noisy speech recognition.