2026-09-18T19:46:36Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/879412021-07-05T14:03:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
James R. Glass.
Alhanai, Tuka (Tuka Waddah Talib Ali Al Hanai)
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2014-06-13T22:34:48Z
2014-06-13T22:34:48Z
2014
2014
http://hdl.handle.net/1721.1/87941
880382239
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2014.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 69-72).
Arabic is a morphologically rich language which rarely displays diacritics. These two features of the language pose challenges when building Automatic Speech Recognition (ASR) systems. Morphological complexity leads to many possible combinations of stems and affixes to form words, and produces texts with high Out Of Vocabulary (OOV) rates. In addition, texts rarely display diacritics which informs the reader about short vowels, geminates, and nunnations (word ending /n/). A lack of diacritics means that 30% of textual information is missing, causing ambiguities in lexical and language modeling when attempting to model pronunciations, and the context of a particular pronunciation. Intuitively, from an English centric view, the phrase th'wrtr wrt n thwrt with 'morphological decomposition' is realized as, th wrtr wrt n th wrt. Including 'diacritics' produces, the writer wrote in the writ. Thus our investigations in this thesis are twofold. Firstly, we show the benefits and interactions between modeling all classes of diacritics (short vowels, geminates, nunnations) in the lexicon. On a Modern Standard Arabic (MSA) corpus of broadcast news, this provides a 1.9% absolute improvement in Word Error Rate (WER) (p < 0.001). We also extend this graphemic lexicon with pronunciation rules, yielding a significant improvement over a lexicon that does not explicitly nodel diacritics. This results in a of 2.4% absolute improvement in WER (p < 0.001). Secondly, we show the benefits of language modeling at the morphemic level with diacritics, over the commonly available, word-based, nondiacratized text. This yields an absolute WER improvement of 1.0% (p < 0.001).
by Tuka Al Hanai.
S.M.
72 pages
eng
Massachusetts Institute of Technology
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Lexical and Language Modeling of Diacritics and Morphemes in Arabic Automatic Speech Recognition
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="0edf0128-58b9-4f12-9e49-7e945a36e275">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Lexical and Language Modeling of Diacritics and Morphemes in Arabic Automatic Speech Recognition</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2014</PublicationDate>
<Authors>
<Author>
<DisplayName>Alhanai, Tuka (Tuka Waddah Talib Ali Al Hanai)</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>Arabic is a morphologically rich language which rarely displays diacritics. These two features of the language pose challenges when building Automatic Speech Recognition (ASR) systems. Morphological complexity leads to many possible combinations of stems and affixes to form words, and produces texts with high Out Of Vocabulary (OOV) rates. In addition, texts rarely display diacritics which informs the reader about short vowels, geminates, and nunnations (word ending /n/). A lack of diacritics means that 30% of textual information is missing, causing ambiguities in lexical and language modeling when attempting to model pronunciations, and the context of a particular pronunciation. Intuitively, from an English centric view, the phrase th'wrtr wrt n thwrt with 'morphological decomposition' is realized as, th wrtr wrt n th wrt. Including 'diacritics' produces, the writer wrote in the writ. Thus our investigations in this thesis are twofold. Firstly, we show the benefits and interactions between modeling all classes of diacritics (short vowels, geminates, nunnations) in the lexicon. On a Modern Standard Arabic (MSA) corpus of broadcast news, this provides a 1.9% absolute improvement in Word Error Rate (WER) (p < 0.001). We also extend this graphemic lexicon with pronunciation rules, yielding a significant improvement over a lexicon that does not explicitly nodel diacritics. This results in a of 2.4% absolute improvement in WER (p < 0.001). Secondly, we show the benefits of language modeling at the morphemic level with diacritics, over the commonly available, word-based, nondiacratized text. This yields an absolute WER improvement of 1.0% (p < 0.001).</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>