Language model rescoring, especially neural network language model (NNLM) rescoring, is widely used to achieve improved performance in a second-pass automatic speech recognition (ASR) system. The rescoring NNLM is usually trained separately from the ASR system. Typically, the two’s training corpora are different, leading to the vocabulary mismatch problem, consequently degrading ASR performance. Previous research focuses more on the language domain mismatch problem, while the vocabulary mismatch problem, which may also cause significant performance degradation, has not been well studied. This paper proposes a novel class-based NNLM framework to address the vocabulary mismatch problem for language model rescoring. Here, OOV words (unknown words to the rescoring NNLM are called OOV words for short) are assigned to well-trained classes of NNLM and inherit the class probability. Experiments show that class-based NNLM rescoring can significantly reduce performance degradation due to vocabulary mismatch.
Cite as: Dai, L., Liu, Q., Yu, K. (2021) Class-Based Neural Network Language Model for Second-Pass Rescoring in ASR. Proc. Interspeech 2021, 2022-2026, doi: 10.21437/Interspeech.2021-1080
@inproceedings{dai21b_interspeech,
author={Lingfeng Dai and Qi Liu and Kai Yu},
title={{Class-Based Neural Network Language Model for Second-Pass Rescoring in ASR}},
year=2021,
booktitle={Proc. Interspeech 2021},
pages={2022--2026},
doi={10.21437/Interspeech.2021-1080}
}