2026-09-19T05:00:44Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1227462021-07-05T14:03:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Mohammad Alizadeh. Addanki, Ravichandra. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2019-11-04T20:22:16Z 2019-11-04T20:22:16Z 2019 2019 https://hdl.handle.net/1721.1/122746 1124766986 Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 Cataloged from PDF version of thesis. Includes bibliographical references (pages 47-50). We present Placeto, a reinforcement learning (RL) approach to efficiently find device placements for distributed neural network training. Unlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph. We propose two key ideas in our approach: (1) we represent the policy as performing iterative placement improvements, rather than outputting a placement in one shot; (2) we use graph embeddings to capture relevant information about the structure of the computation graph, without relying on node labels for indexing. These ideas allow Placeto to train efficiently and generalize to unseen graphs. Our experiments show that Placeto requires up to 6.1 x fewer training steps to find placements that are on par with or better than the best placements found by prior approaches. Moreover, Placeto is able to learn a generalizable placement policy for any given family of graphs, which can then be used without any retraining to predict optimized placements for unseen graphs from the same family. This eliminates the large overhead incurred by prior RL approaches whose lack of generalizability necessitates re-training from scratch every time a new graph is to be placed. by Ravichandra Addanki. S.M. S.M. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science 50 pages eng Massachusetts Institute of Technology MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. Learning generalizable device placement algorithms for distributed machine learning Thesis application/pdf 2019-11-04T20:22:14Z Publication Master EECS unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="16ed04bc-c670-43b6-83f2-3402073c71e8"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>Learning generalizable device placement algorithms for distributed machine learning</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2019</PublicationDate> <Authors> <Author> <DisplayName>Addanki, Ravichandra.</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>We present Placeto, a reinforcement learning (RL) approach to efficiently find device placements for distributed neural network training. Unlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph. We propose two key ideas in our approach: (1) we represent the policy as performing iterative placement improvements, rather than outputting a placement in one shot; (2) we use graph embeddings to capture relevant information about the structure of the computation graph, without relying on node labels for indexing. These ideas allow Placeto to train efficiently and generalize to unseen graphs. Our experiments show that Placeto requires up to 6.1 x fewer training steps to find placements that are on par with or better than the best placements found by prior approaches. Moreover, Placeto is able to learn a generalizable placement policy for any given family of graphs, which can then be used without any retraining to predict optimized placements for unseen graphs from the same family. This eliminates the large overhead incurred by prior RL approaches whose lack of generalizability necessitates re-training from scratch every time a new graph is to be placed.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>