GIM: Guidance as Initialization Method
Name
duitamacortes-jduitama-meng-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
7.97 MB
Format
Adobe PDF
Checksum (MD5)
12bcc157ef6f6800e308dbed61206d4d
Author(s)
Duitama Cortes, Juan Sebastian
Advisor(s)
Katz, Boris
Date Issued
February 2025
Publisher
Massachusetts Institute of Technology
Abstract
This work makes two contributions: the evaluation of early stop guidance for deep Fully Connected Networks (FCNs) and the introduction of guidance as an initialization method (GIM). Network initialization has been a meaningful and challenging topic in the field of machine learning (ML) for a long time. Many initialization methods exist, ranging from data-independent to data-dependent approaches. Initializations allow for a better understanding of model behavior and improvements in model performance. The novel guidance tool enabled us to propose GIM, a new technique that initializes a model by leveraging representational similarity with respect to models of different architectures. A model with an architecture that performs poorly in a specific task can be initialized with guidance from a model with an architecture that performs well in the respective task. We focus on the case of FCNs in the task of image classification and provide experimental results to validate our approach.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
