Automatic dottization of Arabic text (Rasms) using deep recurrent neural networks

Highlights

• First work on automatically adding dots to Arabic text without dots (i.e., Rasms).

• Automatic adding dots using deep recurrent neural networks.

• Evaluated on four different text corpora.

• CER rates ranging from 2.0% to 5.5% on independent test sets.

Abstract

Arabic letters in their early stages were only shapes (Rasm) without dots. Dots were added later to ease reading and reduce ambiguity. Thereafter, diacritics were introduced for phonetic guidance, mainly for nonnative speakers. Many studies have been conducted to automatically diacritize Arabic texts using machine learning techniques. However, to the best of our knowledge, automatically adding dots to Arabic Rasms has not been reported in the literature. In this work, we present the automatic addition of dots to Arabic Rasms using deep recurrent neural networks. Different design choices were explored, including the use of character sequences and word sequences as tokens. The presented techniques were evaluated on four diverse publicly available datasets. Character-level models with stacked BiGRU architecture outperformed all the other architectures with character error rates ranging from 2.0% to 5.5% and dottization error rates ranging from 4.2% to 11.0% on independent test sets.

Introduction

Languages and scripts evolve over time. In the early stage of the Arabic language, the alphabet characters were only shapes (called Rasm) without dots or diacritics. Dots were added to the characters to differentiate between similar looking characters. Subsequently, the diacritics were invented and are used for resolving ambiguities and for phonetic guidance. Nowadays, dots are used permanently with characters, unlike diacritics which are used in limited circumstances. Fig. 1 illustrates this idea with an example Arabic sentence written once using only the Rasms, once with the addition of dots, and once with the addition of diacritics along with dots. As a second example, an image of a parchment having an ancient Quranic manuscript written on it is shown in Fig. 2. It was written using Rasms and does not contain any dots.¹

A Rasm sequence without dots can represent several possible words. Fig. 3 presents two example Rasm sequences and different possible Arabic words they can represent. For the first example (the top row in the figure), we have six possible words a Rasm sequence can represent. Similarly, for the second example, the presented Rasm sequence can represent five different words. The correct word depends on the context of the text and it is difficult to identify instantly, even by a native speaker.

Many studies have been performed to automate the diacritization process, Abandah et al. [1], Metwally et al. [2], Masmoudi et al. [3]. Moreover, Arabic diacritization is beneficial to other tasks in Arabic natural language processing (ANLP), such as translation Diab et al. [4]. Since dots were introduced before diacritics and are more fundamental to the script, it motivated us to investigate the use of deep learning for automatic dottization of Arabic texts. We believe that automatic dottization will contribute to future ANLP research both directly and indirectly. A direct use case of automatic dottization can be to automatically transcribe ancient Arabic manuscripts once Rasms have been recognized. A possible implication of this research can be on Arabic text recognition research. Some researchers have investigated the idea of splitting the task of Arabic text recognition into multiple stages involving recognizing Arabic Rasms separately from dots and then combining the Rasms and dots to output the final Arabic texts Ahmad and Fink [[5], [6]]. Another use of this approach could be in the tokenization of Arabic texts for other NLP tasks. Additionally, a relatively newer use case is in the field of social media moderation, where users use Arabic texts without dots to evade censorship. As an example, Fig. 4 shows an Arabic tweet using only Rasms.

To the best of our knowledge, automatically adding dots to Arabic Rasms has not been reported in the literature. In this paper, we will present the automatic addition of dots to Arabic text using deep recurrent neural networks. We present two different approaches to dottization, one using word sequences as input and the other using character sequences as input. The presented techniques were evaluated on four different Arabic text corpora that are publicly available.

It should be noted that although automatic Arabic diacritization is a related problem to automatic Arabic dottization, they have differences at the same time. Dots are more fundamental in Arabic texts compared to the diacritics. Most Arabic texts are written without diacritics but not without dots. The challenges and opportunities involved in adding dots are different from those involved in adding diacritics. Although a character can take any diacritic from among the possible diacritics in Arabic, a sequence of diacritics over a word is very limited. Nouns, for example, have a fixed sequence of diacritics, and they do not change except the diacritic on the last character. Furthermore, the sequence of diacritics over a verb follows a certain template, such as fatha fatha fatha as in

or damma kasra fatha as in

and they cannot appear in every possible combination. Additionally, many of the diacritics can appear only at specific positions in a word. For example, the diacritic sukun cannot come over the first character of a word. Similarly, tanwin diacritics can only appear over the last character of a word. On the other hand, dots are fundamental in defining the characters themselves. They do not follow a fixed template as diacritics. Additionally, some Arabic Rasms belong to a single character (like ا and ل) and do not have dots. Moreover, some word Rasms lend themselves to only a specific word in Arabic, as no other Arabic word appears with the same Rasm sequence. This insight can help in the dottization task, as we will discuss in the rest of the paper. Furthermore, homographs in Arabic can have different diacritics, but dottization does not face such issues. For example, the Arabic word

can mean the noun gold

or the verb he went

depending on the diacritic over the last character. On the other hand, many verb forms, such as

(he writes) and

(she writes) and

(we write) have the same Rasm sequence but different dots in the prefix, but all the forms have the same diacritic sequence, fatha sukun damma fatha.

The rest of the paper is structured as follows: in Section 2, we present the background on the Arabic script. In Section 3, we present the related work specifically focusing on Arabic diacritization. In Section 4, we present the methodology, including tokenization, model design and postprocessing steps. In Section 5, we present the datasets used for experimentation, the evaluation metrics used to report the results, the experimentation and results and some discussions related to error analysis. Finally, in Section 6, we conclude with our findings and discuss possible extensions to our work.