Feature engineering and evaluation in lightweight systems
Name
1129235741-MIT.pdf
Size
2.07 MB
Format
Adobe PDF
Checksum (MD5)
b8da8442ebb5cfa0fd084e9dfeae7330
Author(s)
Lu, Kelvin,M. Eng.Massachusetts Institute of Technology.
Advisor(s)
Kalyan Veeramachaneni.
Date Issued
2019
Publisher
Massachusetts Institute of Technology
Abstract
This thesis presents Ballet, a lightweight, feature engineering framework that allows users to contribute to an open-source data science project. Specifically, it provides a framework for users to easily write flexible and high-quality features from a raw dataset. In addition, it provides a series of tests to ensure that all features in a project follow a consistent API and all provide some level of predictive power towards a target column. For the latter task, we modified and implemented GFSSF, a feature selection algorithm designed specifically for grouped, streaming features. This included building a performant entropy and mutual information estimator for datasets, as well as integrating this algorithm into Travis Cl, a popular continuous integration tool. We then evaluated our framework on a popular test dataset for data scientists, evaluating for performance and ease of development.
Description
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 69-70).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
Persistent DSpace Link
