ISCA Archive Interspeech 2021
ISCA Archive Interspeech 2021

A Weight Moving Average Based Alternate Decoupled Learning Algorithm for Long-Tailed Language Identification

Hui Wang, Lin Liu, Yan Song, Lei Fang, Ian McLoughlin, Li-Rong Dai

Language identification (LID) research has made tremendous progress in recent years, especially with the introduction of deep learning techniques. However, for real-world applications where the distribution of different language data is highly imbalanced, the performance of existing LID systems is still far from satisfactory. This raises the challenge of long-tailed LID. In this paper, we propose an effective weight moving average (WMA) based alternate decoupled learning algorithm, termed WADCL, for long-tailed LID. The system is divided into two components, a frontend feature extractor and a backend classifier. These are then alternately learned in an end-to-end manner using different sampling schemes to alleviate the distribution mismatch between training and test datasets. Furthermore, our WMA method aims to mitigate the side-effects of re-sampling schemes, by fusing the model parameters learned along the trajectory of stochastic gradient descent (SGD) optimization. To validate the effectiveness of the proposed WADCL algorithm, we evaluate and compare several systems over a language dataset constructed to match a long-tailed distribution based on real world application [1]. The experimental results from the long-tailed language dataset demonstrate that the proposed algorithm is able to achieve significant performance gains over existing state-of-the-art x-vector based LID methods.


doi: 10.21437/Interspeech.2021-776

Cite as: Wang, H., Liu, L., Song, Y., Fang, L., McLoughlin, I., Dai, L.-R. (2021) A Weight Moving Average Based Alternate Decoupled Learning Algorithm for Long-Tailed Language Identification. Proc. Interspeech 2021, 1499-1503, doi: 10.21437/Interspeech.2021-776

@inproceedings{wang21o_interspeech,
  author={Hui Wang and Lin Liu and Yan Song and Lei Fang and Ian McLoughlin and Li-Rong Dai},
  title={{A Weight Moving Average Based Alternate Decoupled Learning Algorithm for Long-Tailed Language Identification}},
  year=2021,
  booktitle={Proc. Interspeech 2021},
  pages={1499--1503},
  doi={10.21437/Interspeech.2021-776}
}