2026-09-18T17:32:24Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1630412025-10-07T04:14:17Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Amarasinghe, Saman Rajvee, Muhender Raj Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2025-10-06T17:40:59Z 2025-10-06T17:40:59Z 2025-05 2025-06-23T14:03:24.188Z https://hdl.handle.net/1721.1/163041 With the rise of large language models, there have been efforts to optimize machine learning inference to support a large volume of queries. Currently, the two main ways to do this are running optimized kernels for computing the forward inference pass and distributing computation across multiple GPUs or different cores in a GPU. Machine learning libraries such as PyTorch produce dynamic computation graphs in order to represent the forward pass of the model. PyTorch allows conversion of these dynamic graphs into static ones through just-in-time (JIT) compilation. These graphs can then be optimized further by the compiler. We propose an alternate way of optimizing these dynamic graphs. We convert the dynamic computation graph of PyTorch to pipelines in StreamIt, a domain specific language (DSL) for streaming applications, and use the multi-stage compilation property of BuildIt to compile this pipeline in stages to inference code. We found that, while the inference latencies of models compiled in this way are slightly higher, they are still comparable to those of PyTorch models and are open to future optimizations. M.Eng. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ Converting PyTorch Models to StreamIt Pipelines Thesis application/pdf Master Master of Engineering in Electrical Engineering and Computer Science Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="7c2674a9-26aa-4622-9b1f-9b846592e7b4"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Converting PyTorch Models to StreamIt Pipelines</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-05</PublicationDate> <Authors> <Author> <DisplayName>Rajvee, Muhender Raj</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>With the rise of large language models, there have been efforts to optimize machine learning inference to support a large volume of queries. Currently, the two main ways to do this are running optimized kernels for computing the forward inference pass and distributing computation across multiple GPUs or different cores in a GPU. Machine learning libraries such as PyTorch produce dynamic computation graphs in order to represent the forward pass of the model. PyTorch allows conversion of these dynamic graphs into static ones through just-in-time (JIT) compilation. These graphs can then be optimized further by the compiler. We propose an alternate way of optimizing these dynamic graphs. We convert the dynamic computation graph of PyTorch to pipelines in StreamIt, a domain specific language (DSL) for streaming applications, and use the multi-stage compilation property of BuildIt to compile this pipeline in stages to inference code. We found that, while the inference latencies of models compiled in this way are slightly higher, they are still comparable to those of PyTorch models and are open to future optimizations.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>