2026-09-19T07:10:26Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1135402026-06-06T00:54:34Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Samuel Madden. Viswanathan, Srinidhi Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2018-02-08T16:28:19Z 2018-02-08T16:28:19Z 2017 2017 http://hdl.handle.net/1721.1/113540 1020175819 Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017. Cataloged from PDF version of thesis. Includes bibliographical references (pages 99-100). Building a machine learning model is often an iterative process. Data scientists train hundreds of models before finding a model that meets acceptable criteria. But tracking these models and remembering the insights obtained from them is an arduous task. In this thesis, we present two main systems for facilitating better tracking, analysis, and querying of scikit-learn machine learning models. First, we introduce our scikit-learn client for ModelDB, a novel end-to-end system for managing machine learning models. The client allows data scientists to easily track diverse scikit-learn workflows with minimal changes to their code. Then, we describe our extension to ModelDB, PredictionStore. While the ModelDB client enables users to track the different models they have run, PredictionStore creates a prediction matrix to tackle the remaining piece in the puzzle: facilitating better exploration and analysis of model performance. We implement a query API to assist in analyzing predictions and answering nuanced questions about models. We also implement a variety of algorithms to recommend particular models to ensemble utilizing the prediction matrix. We evaluate ModelDB and PredictionStore on different datasets and determine ModelDB successfully tracks scikit-learn models, and most complex model queries can be executed in a matter of seconds using our query API. In addition, the workflows demonstrate significant improvement in accuracy using the ensemble algorithms. The overall goal of this research is to provide a flexible framework for training scikit-learn models, storing their predictions/ models, and efficiently exploring and analyzing the results. by Srinidhi Viswanathan. M.Eng. 100 pages eng Massachusetts Institute of Technology MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. ModelDB : tools for machine learning model management and prediction storage Tools for machine learning model management and prediction storage Thesis application/pdf false Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="6805de18-4093-4bec-a3f3-0bbadbda3cd2"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>ModelDB : tools for machine learning model management and prediction storage</Title> <Subtitle>Tools for machine learning model management and prediction storage</Subtitle> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2017</PublicationDate> <Authors> <Author> <DisplayName>Viswanathan, Srinidhi</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>Building a machine learning model is often an iterative process. Data scientists train hundreds of models before finding a model that meets acceptable criteria. But tracking these models and remembering the insights obtained from them is an arduous task. In this thesis, we present two main systems for facilitating better tracking, analysis, and querying of scikit-learn machine learning models. First, we introduce our scikit-learn client for ModelDB, a novel end-to-end system for managing machine learning models. The client allows data scientists to easily track diverse scikit-learn workflows with minimal changes to their code. Then, we describe our extension to ModelDB, PredictionStore. While the ModelDB client enables users to track the different models they have run, PredictionStore creates a prediction matrix to tackle the remaining piece in the puzzle: facilitating better exploration and analysis of model performance. We implement a query API to assist in analyzing predictions and answering nuanced questions about models. We also implement a variety of algorithms to recommend particular models to ensemble utilizing the prediction matrix. We evaluate ModelDB and PredictionStore on different datasets and determine ModelDB successfully tracks scikit-learn models, and most complex model queries can be executed in a matter of seconds using our query API. In addition, the workflows demonstrate significant improvement in accuracy using the ensemble algorithms. The overall goal of this research is to provide a flexible framework for training scikit-learn models, storing their predictions/ models, and efficiently exploring and analyzing the results.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>