Automatic sub-word unit discovery and pronunciation lexicon induction for ASR with application to under-resourced languages

Abstract

We present a method enabling the unsupervised discovery of sub-word units (SWUs) and associated pronunciation lexicons for use in automatic speech recognition (ASR) systems. This includes a novel SWU discovery approach based on self-organising HMM-GMM states that are agglomeratively tied across words as well as a novel pronunciation lexicon induction approach that iteratively reduces pronunciation variation by means of model pruning. Our approach relies only on recorded speech and associated orthographic transcriptions and does not require alphabetic graphemes. We apply our methods to corpora of recorded radio broadcasts in Ugandan English, Luganda and Acholi, of which the latter two are under-resourced. The speech is conversational and contains high levels of background noise, and therefore presents a challenge to automatic lexicon induction. We demonstrate that our proposed method is able to discover lexicons that perform as well as baseline expert systems for Acholi, and close to this level for the other two languages when used to train DNN-HMM ASR systems. This demonstrates the potential of the method to enable and accelerate ASR for under-resourced languages for which a phone inventory and pronunciation lexicon are not available by eliminating the dependence on human expertise this usually requires.

Introduction

We present a set of approaches for the development of automatic speech recognition (ASR) systems in an under-resourced setting. We constrain ourselves to using a training corpus that consist only of recorded speech and associated orthographic transcriptions and where a phone set and lexicon are not available. Although we demonstrate that the incorporation of graphemic knowledge can improve the performance of our systems, we do not rely on the availability of alphabetic graphemes. As such, the approaches described here should in principle be suited to languages that use a logographic orthography.

The development of sub-word unit inventories and associated pronunciations is generally a time consuming and expensive process which requires the availability of linguists that are familiar with the task language. If the steps of sub-word unit discovery and pronunciation lexicon generation could be automated with satisfactory performance, then it would greatly streamline the process of implementing ASR in an under-resourced setting. In some cases (e.g. where trained linguists are not available) it would enable the implemention of ASR where it would otherwise be infeasible. It may even be possible that such a data-driven approach to lexicon generation could improve ASR performance even for well resourced tasks, but we do not assess that here.

The main objective of this work is to demonstrate the feasibility of performing ASR with an automatically induced pronunciation lexicon in a truly under-resourced setting. Many studies have addressed parts of this problem, but these typically focus only on unit segmentation and discovery, while neglecting the subsequent development of pronunciation lexicons to perform ASR. Other studies have focused on designing lexicons, but have typically adapted pre-existing phonemes or alphabetic graphemes for use as sub-word units (SWU’s). In the studies that have applied automated pronunciation lexicon design with automatically discovered SWU’s to ASR, the datasets used are low-perplexity (e.g. connected digits or weather query corpora) or represent ideal scenarios such as read speech that may not be representative of the results that would be obtained in an under-resourced large-vocabulary continuous speech task. In contrast, we will show that both fully automatic SWU discovery and lexicon induction of a sufficiently high quality is feasible even in the challenging scenario in which the data consists of low-quality recordings of large-vocabulary spontaneous speech and where no graphemic information is used. To this end, we present a new SWU discovery approach based on self-organising HMM-GMM states that are agglomeratively tied across words. In addition, we present a novel pronunciation lexicon induction method that models variation using an HMM that generates discrete SWU sequences and that iteratively reduces pronunciation variation by pruning the model’s state emission distributions.