MIT Libraries homeMIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Language-independent methods for computer-assisted pronunciation training

Author(s)
Lee, Ann, Ph. D. Massachusetts Institute of Technology
Thumbnail
DownloadFull printable version (15.95Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
James Glass.
Terms of use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
Computer-assisted pronunciation training (CAPT) systems help students practice speaking foreign languages by providing automatic pronunciation assessment and corrective feedback. Automatic speech recognition (ASR) technology is a natural component in CAPT systems. Since a nonnative speaker's native language (Li) background affects their pronunciation patterns in a target language (L2), typically not only native but also nonnative training data of specific Ls is needed to train a recognizer for CAPT systems. Given that there are around 7,000 languages in the world, the data collection process is costly and has scalability issues. In addition, expert knowledge on the target L2 is also often needed to design a large feature set describing the deviation of nonnative speech from native speech. In contrast to machines, it is relatively easy for native listeners to detect pronunciation errors without being exposed to nonnative speech or trained with linguistic knowledge beforehand. In this thesis, we are interested in this unsupervised capability and propose methods to overcome the language-dependent challenges. Inspired by the success of unsupervised acoustic pattern discovery, we propose to discover an individual learner's pronunciation error patterns in an unsupervised manner by analyzing the acoustic similarity between speech segments from the learner. Experimental results on nonnative English and nonnative Mandarin Chinese spoken by students from different Ls show that the proposed method is Li-independent and can be portable to different L2s. Moreover, the method is personalized such that it accommodates variations in pronunciation patterns across students. In addition, motivated by the success of deep learning models in unsupervised feature learning, we explore the use of convolutional neural networks (CNNs) for mispronunciation detection. A language-independent data augmentation method is developed to take advantage of native speech as training samples. Experimental results on nonnative Mandarin Chinese speech show the effectiveness of the model and the method. Moreover, both qualitative and quantitative analyses on the convolutional filters reveal that the CNN automatically learns a set of human-interpretable high-level features.
Description
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2016.
 
Cataloged from PDF version of thesis.
 
Includes bibliographical references (pages 137-145).
 
Date issued
2016
URI
http://hdl.handle.net/1721.1/107338
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries homeMIT Libraries logo

Find us on

Twitter Facebook Instagram YouTube RSS

MIT Libraries navigation

SearchHours & locationsBorrow & requestResearch supportAbout us
PrivacyPermissionsAccessibility
MIT
Massachusetts Institute of Technology
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.