2026-09-21T06:13:42Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1591262026-04-28T03:36:00Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Zhang, Bin Jiang, Peng Zheng, Jared Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2025-04-14T14:07:13Z 2025-04-14T14:07:13Z 2025-02 2025-04-03T14:06:38.137Z https://hdl.handle.net/1721.1/159126 In recent years, transformer architectures have shown remarkable capabilities in learning meaningful representations from text and images. This approach has been extended to the realm of protein sequences through pretrained protein language models, which have excelled in various protein engineering tasks. In this thesis, we investigate a pre-trained protein language model’s ability to predict protein structure and the effects of mutations. For many advanced protein understanding tasks, such as predicting protein function and protein-protein interactions, fine-tuning of the model is essential. We explore methods to fine-tune the Evolutionary Scale Modeling (ESM2) model, a pretrained protein language model, for predicting protein functions structured as Gene Ontology terms and predicting protein-protein interactions. Notably, we develop a novel method of modeling the hierarchy constraint in GO term prediction that improves training convergence and test performance while making the model hierarchically consistent with GO. This research aims to enhance our understanding of protein language models in decoding complex biological information, thereby contributing to advancements in computational biology. M.Eng. Massachusetts Institute of Technology Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ Structure, Function, and Interaction in Protein Language Models Thesis application/pdf Master Master of Engineering in Computer Science and Molecular Biology Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="6c549805-1c3b-4cd5-88c6-0995aa4c97c1"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Structure, Function, and Interaction in Protein Language Models</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-02</PublicationDate> <Authors> <Author> <DisplayName>Zheng, Jared</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://creativecommons.org/licenses/by-nc-nd/4.0/</License> <Abstract>In recent years, transformer architectures have shown remarkable capabilities in learning meaningful representations from text and images. This approach has been extended to the realm of protein sequences through pretrained protein language models, which have excelled in various protein engineering tasks. In this thesis, we investigate a pre-trained protein language model’s ability to predict protein structure and the effects of mutations. For many advanced protein understanding tasks, such as predicting protein function and protein-protein interactions, fine-tuning of the model is essential. We explore methods to fine-tune the Evolutionary Scale Modeling (ESM2) model, a pretrained protein language model, for predicting protein functions structured as Gene Ontology terms and predicting protein-protein interactions. Notably, we develop a novel method of modeling the hierarchy constraint in GO term prediction that improves training convergence and test performance while making the model hierarchically consistent with GO. This research aims to enhance our understanding of protein language models in decoding complex biological information, thereby contributing to advancements in computational biology.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>