Language-independent methods for computer-assisted pronunciation training
Name
972906719-MIT.pdf
Description
Full printable version
Size
15.96 MB
Format
Adobe PDF
Checksum (MD5)
b724c946671167455b41fc0f27b5818a
Author(s)
Lee, Ann, Ph. D. Massachusetts Institute of Technology
Advisor(s)
James Glass.
Date Issued
2016
Publisher
Massachusetts Institute of Technology
Abstract
Computer-assisted pronunciation training (CAPT) systems help students practice speaking foreign languages by providing automatic pronunciation assessment and corrective feedback. Automatic speech recognition (ASR) technology is a natural component in CAPT systems. Since a nonnative speaker's native language (Li) background affects their pronunciation patterns in a target language (L2), typically not only native but also nonnative training data of specific Ls is needed to train a recognizer for CAPT systems. Given that there are around 7,000 languages in the world, the data collection process is costly and has scalability issues. In addition, expert knowledge on the target L2 is also often needed to design a large feature set describing the deviation of nonnative speech from native speech. In contrast to machines, it is relatively easy for native listeners to detect pronunciation errors without being exposed to nonnative speech or trained with linguistic knowledge beforehand. In this thesis, we are interested in this unsupervised capability and propose methods to overcome the language-dependent challenges. Inspired by the success of unsupervised acoustic pattern discovery, we propose to discover an individual learner's pronunciation error patterns in an unsupervised manner by analyzing the acoustic similarity between speech segments from the learner. Experimental results on nonnative English and nonnative Mandarin Chinese spoken by students from different Ls show that the proposed method is Li-independent and can be portable to different L2s. Moreover, the method is personalized such that it accommodates variations in pronunciation patterns across students. In addition, motivated by the success of deep learning models in unsupervised feature learning, we explore the use of convolutional neural networks (CNNs) for mispronunciation detection. A language-independent data augmentation method is developed to take advantage of native speech as training samples. Experimental results on nonnative Mandarin Chinese speech show the effectiveness of the model and the method. Moreover, both qualitative and quantitative analyses on the convolutional filters reveal that the CNN automatically learns a set of human-interpretable high-level features.
Description
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2016.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 137-145).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
Persistent DSpace Link
