Abstract
This paper presents our approach for automatic speech recognition (ASR) of overlapping speech. Our system consists of two principal components: a speech separation component and a feature estmation component. In the speech separation phase, we first estimated the speaker’s position, and then the speaker location information is used in a GSC-configured beamformer with a minimum mutual information (MMI) criterion, followed by a Zelinski and binary-masking post-filter, to separate the speech of different speakers. In the feature estimation phase, the neural networks are trained to learn the mapping from the features extracted from the pre-separated speech to those extracted from the close-talking microphone speech signal. The outputs of the neural networks are then used to generate acoustic features, which are subsequently used in acoustic model adaptation and system evaluation. The proposed approach is evaluated through ASR experiments on the PASCAL Speech Separation Challenge II (SSC2) corpus. We demonstrate that our system provides large improvements in recognition accuracy compared with a single distant microphone case and the performance of ASR system can be significantly improved both through the use of MMI beamforming and feature mapping approaches.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Haykin, S.: Unsupervised adaptive filtering. Blind source seperation, vol. 1. Wiley, New York (2000)
Frost, O.L.: An algorithm for linearly constrained adaptive array processing. Proc. IEEE 60(8), 926–935 (1972)
Griffiths, L.J., Jim, C.W.: An Alternative Approach to Linearly Constrained Adaptive Beamforming. IEEE Trans. on Antennas and Propagation AP-30(1), 27–34 (1982)
Moore, D., McCowan, I.: Microphone array speech recognition: Experiments on overlapping speech in meetings. In: Proc. ICASSP, pp. 497–500 (2003)
Stolcke, A., et al.: The SRI-ICSI Spring 2007 Meeting and Lecture Recognition System. LNCS. Springer, Heidelberg (2007)
Cetin, O., Shriberg, E.: Speaker overlaps and ASR errors in meetings: Effects before, during, and after the overlap. In: Proc. ICASSP, vol. 1, pp. 357–360 (2006)
Hain, T., Burget, L., Dines, J., Garau, G., Wan, V., Karafiat, M., Vepa, J., Lincoln, M.: The AMI system for the transcription of speech in meetings. In: Proc. ICASSP, Honolulu, Hawaii (2007)
Li, W., Magimai.-Doss, M., Dines, J., Bourlard, H.: MLP-based log spectral energy mapping for robust overlapping speech recognition, IDIAP Technical Report, 07-54 (2007)
Young, S., et al.: The HTK Book, Version 3.4., http://htk.eng.cam.ac.uk/docs/docs.shtml
Li, W., Dines, J., Magimai.-Doss, M., Bourlard, H.: Neural Network based Regression for Robust Overlapping Speech Recognition using Microphone Arrays, IDIAP Technical Report 08-09 (2008)
Lincoln, M., McCowan, I., Vepa, I., Maganti, H.K.: The multichannel Wall Street Journal audio visual corpus ( mc-wsj-av): Specification and initial experiments. In: Proc. ASRU, pp. 357–362 (2005)
Gehrig, T., McDonough, J.: Tracking and far-field speech recognition for multiple simultaneous speakers. In: Proc. the Workshop on Machine Learning and Multimodal Interaction (September 2006)
Kumatani, K., Gehrig, T., Mayer, U., Stoimenov, E., McDonough, J., Wölfel, M.: Adaptive beamforming with a minimum mutual information criterion. IEEE Transactions on Audio, Speech and Language Processing 15, 2527–2541 (2007)
Buchner, H., Aichner, R., Kellermann, W.: Blind source seperation for convolutive mixtures: A unified treatment. In: Audio Signal Processing for Next-Generation Multimedia Communication Systems, pp. 255–289. Kluwer Academic, Boston (2004)
Van Trees, H.L.: Optimum Array Processing. Wiley-Interscience, New York (2002)
Werbos, P.J.: Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, PhD Thesis. Harvard University, Cambridge, MA (1974)
Sorensen, H.B.D.: A cepstral noise reductionmulti-layer neural network. In: Proc. ICASSP, vol. 2, pp. 933–936 (1991)
Yuk, D., Flanagan, J.: Telephone speech recognition using neural networks and hiddenMarkov models. In: Proc. ICASSP, vol. 1, pp. 157–160 (1999)
Che, C., Lin, Q., Pearson, J., de Vries, B., Flanagan, J.: Microphone arrays and neural networks for robust speech recognition. In: Proc. the workshop on Human Language Technology, pp. 342–347 (1994)
Uwe Simmer, K., Bitzer, J., Marro, C.: Post-filtering techniques. In: Brandstein, M., Ward, D. (eds.) Microphone Arrays, ch. 3, pp. 39–60. Springer, Heidelberg (2001)
McCowan, I., Hari-Krishna, M., Gatica-Perez, D., Moore, D., Ba, S.: Speech Acquisition in Meetings with an Audio-Visual Sensor Array. In: Proc. the IEEE International Conference on Multimedia and Expo (ICME) (July 2005)
Rabiner, L.R., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall, Englewood Cliffs (1993)
Lin, Q., Che, C., Yuk, D.-S., Jin, L., de Vries, B., Pearson, J., Flanagan, J.: Robust distant-talking speech recognition. In: Proc. ICASSP, vol. 1, pp. 21–24 (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, W., Kumatani, K., Dines, J., Magimai-Doss, M., Bourlard, H. (2008). A Neural Network Based Regression Approach for Recognizing Simultaneous Speech. In: Popescu-Belis, A., Stiefelhagen, R. (eds) Machine Learning for Multimodal Interaction. MLMI 2008. Lecture Notes in Computer Science, vol 5237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85853-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-85853-9_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85852-2
Online ISBN: 978-3-540-85853-9
eBook Packages: Computer ScienceComputer Science (R0)