Long-range Genomics Benchmark Technology and More
Name
polen-mpolen-meng-eecs-2024-thesis.pdf
Description
Thesis PDF
Size
2.48 MB
Format
Adobe PDF
Checksum (MD5)
30748eef461f120d727a1c650368d94e
Author(s)
Polen, McKinley
Advisor(s)
Kellis, Manolis
Date Issued
May 2024
Publisher
Massachusetts Institute of Technology
Abstract
The transformer architecture has emerged as a popular choice in various domains, owing to its ability to capture long-range dependencies and parallel processing capabilities. In the context of genomics, where dependencies often span over 100,000 base pairs, the quadratic computational complexity of the attention mechanism, a core feature of the transformer architecture, poses a significant bottleneck. With the goal of creating a genomics foundation model (FM), this paper aims to address challenges associated long range dependencies in genomics. Our survey encompasses modifications to the attention mechanism, the creation of a genomics long range benchmark (GLRB), and the evaluation of various transformer and other non-transformer architectures. These efforts collectively develop the groundwork supporting the development of a robust genomics foundation model, opening new possibilities for genomics research and applications.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
