Termite: a system for tunneling through heterogeneous data
Name
1903.05008.pdf
Description
Accepted version
Size
1.11 MB
Format
Unknown
Checksum (MD5)
64b109c1e741f476e3e0ff68b29cd6fc
Author(s)
Castro Fernandez, Raul
Madden, Samuel R
Date Issued
2019
Journal
Proceedings of the ACM SIGMOD International Conference on Management of Data
Publisher
Association for Computing Machinery (ACM)
Citation
Fernandez, Raul Castro and Madden, Samuel. 2019. "Termite: a system for tunneling through heterogeneous data." Proceedings of the ACM SIGMOD International Conference on Management of Data.
Version
Author's final manuscript
Abstract
© 2019 ACM. Data-driven analysis is important in virtually every modern organization. Yet, most data is underutilized because it remains locked in silos inside of organizations; large organizations have thousands of databases, and billions of files that are not integrated together in a single, queryable repository. Despite 40+ years of continuous effort by the database community, data integration still remains an open challenge. In this paper, we advocate a different approach: rather than trying to infer a common schema, we aim to find another common representation for diverse, heterogeneous data. Specifically, we argue for an embedding (i.e., a vector space) in which all entities, rows, columns, and paragraphs are represented as points. In the embedding, the distance between points indicates their degree of relatedness. We present Termite, a prototype we have built to learn the best embedding from the data. Because the best representation is learned, this allows Termite to avoid much of the human effort associated with traditional data integration tasks. On top of Termite, we have implemented a Termite-Join operator, which allows people to identify related concepts, even when these are stored in databases with different schemas and in unstructured data such as text files, webpages, etc. Finally, we show preliminary evaluation results of our prototype via a user study, and describe a list of future directions we have identified.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3329859.3329877
