Building Data Civilizer Pipelines with an Advanced Workflow Engine
Name
icde2018-demo-civilizer.pdf
Description
Accepted version
Size
346.43 KB
Format
Unknown
Checksum (MD5)
bd120a0ae72b6f1d4e8f09d0a69e2e6d
Author(s)
Mansour, Essam
Deng, Dong
Castro Fernandez, Raul
Qahtan, Abdulhakim A.
Tao, Wenbo
Abedjan, Ziawasch
Elmagarmid, Ahmed
Ilyas, Ihab F.
Madden, Samuel R
Ouzzani, Mourad
Date Issued
April 2018
Publisher
IEEE
Citation
Mansour, Essam, Deng, Dong, Fernandez, Raul Castro, Qahtan, Abdulhakim A., Tao, Wenbo et al. 2018. "Building Data Civilizer Pipelines with an Advanced Workflow Engine."
Version
Author's final manuscript
Abstract
© 2018 IEEE. In order for an enterprise to gain insight into its internal business and the changing outside environment, it is essential to provide the relevant data for in-depth analysis. Enterprise data is usually scattered across departments and geographic regions and is often inconsistent. Data scientists spend the majority of their time finding, preparing, integrating, and cleaning relevant data sets. Data Civilizer is an end-To-end data preparation system. In this paper, we present the complete system, focusing on our new workflow engine, a superior system for entity matching and consolidation, and new cleaning tools. Our workflow engine allows data scientists to author, execute and retrofit data preparation pipelines of different data discovery and cleaning services. Our end-To-end demo scenario is based on data from the MIT data warehouse and e-commerce data sets.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/icde.2018.00184
