2026-09-19T20:26:32Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1180392022-06-27T17:04:38Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Michael Stonebraker. Tao, Wenbo, Ph. D. Massachusetts Institute of Technology Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2018-09-17T15:54:53Z 2018-09-17T15:54:53Z 2018 2018 http://hdl.handle.net/1721.1/118039 1051459082 Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018. Cataloged from PDF version of thesis. Includes bibliographical references (pages 61-63). String joins have wide applications in data integration and cleaning. The inconsistency of data caused by data errors, term variations and missing values has led to the need for approximate string joins (ASJ). In this thesis, we study ASJ with abbreviations, which are a frequent type of term variation. Although prior works have studied ASJ given a user-inputted dictionary of synonym rules, they have three common limitations. First, they suffer from low precision in the presence of abbreviations having multiple full forms. Second, their join algorithms are not scalable due to the exponential time complexity. Third, the dictionary may not exist since abbreviations are highly domain-dependent. We propose an end-to-end workflow to address these limitations. There are three main components in the workflow: (1) a new similarity measure taking abbreviations into account that can handle abbreviations having multiple full forms, (2) an efficient join algorithm following the filter-verification framework and (3) an unsupervised approach to learn a dictionary of abbreviation rules from input strings. We evaluate our workflow on four real-world datasets and show that our workflow outputs accurate join results, scales well as input size grows and greatly outperforms state-of-the-art approaches in both accuracy and efficiency. by Wenbo Tao. S.M. 63 pages eng Massachusetts Institute of Technology MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. Approximate string joins with abbreviations ASJ with abbreviations Thesis application/pdf false Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="25b9153e-2b5e-4db3-a240-e8f8449c2ed6"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>Approximate string joins with abbreviations</Title> <Subtitle>ASJ with abbreviations</Subtitle> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2018</PublicationDate> <Authors> <Author> <DisplayName>Tao, Wenbo, Ph. D. Massachusetts Institute of Technology</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>String joins have wide applications in data integration and cleaning. The inconsistency of data caused by data errors, term variations and missing values has led to the need for approximate string joins (ASJ). In this thesis, we study ASJ with abbreviations, which are a frequent type of term variation. Although prior works have studied ASJ given a user-inputted dictionary of synonym rules, they have three common limitations. First, they suffer from low precision in the presence of abbreviations having multiple full forms. Second, their join algorithms are not scalable due to the exponential time complexity. Third, the dictionary may not exist since abbreviations are highly domain-dependent. We propose an end-to-end workflow to address these limitations. There are three main components in the workflow: (1) a new similarity measure taking abbreviations into account that can handle abbreviations having multiple full forms, (2) an efficient join algorithm following the filter-verification framework and (3) an unsupervised approach to learn a dictionary of abbreviation rules from input strings. We evaluate our workflow on four real-world datasets and show that our workflow outputs accurate join results, scales well as input size grows and greatly outperforms state-of-the-art approaches in both accuracy and efficiency.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>