Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Libraries
  3. MIT Theses
  4. Doctoral Theses
  5. Controlling Generative Models via Representation Learning, Reasoning, and Reward Optimization

Controlling Generative Models via Representation Learning, Reasoning, and Reward Optimization

Thumbnail Image
Download
Name

wang-wangchy-phd-eecs-2026-thesis.pdf

Size

26.25 MB

Format

Adobe PDF

Checksum (MD5)

99d200688297b938fb156189cfe56a27

Author(s)
Wang, Chenyu
Advisor(s)
Jaakkola, Tommi S.
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Generative models, particularly diffusion models and Large Language Models (LLMs), have achieved remarkable success in synthesizing high-fidelity data across continuous and discrete domains. However, standard training paradigms often prioritize distribution matching over structural coherence and functional utility, leading to inefficient training and limited controllability. This thesis addresses these limitations by developing complementary frameworks for guided generation, systematically exploring how the generative process can be directed by structured representations, dynamic reasoning, and extrinsic rewards. First, we explore internal guidance by leveraging the inherent structure of data representations. To start with, effective guidance necessitates a clean signal. However, in multimodal domains, information entanglement and the modality gap often obscure true underlying semantics, making standard representations unreliable for precise control. This motivates our development of a theoretical framework for learning disentangled representations of multimodal data, which rigorously separates shared information from modality-specific one. With this foundation, we leverage structured representations to actively guide generation. In particular, we present REED, a framework that utilizes external pretrained multimodal representations to inject semantic guidance into the denoising process, significantly accelerating diffusion model training. Then, we move beyond static guidance to dynamic reasoning. Specifically, we enhance text diffusion models with HDLM, a framework that imposes a structural reasoning process on token generation. By exploiting the hierarchical nature of token semantics, HDLM guides the model to reason from coarse-grained concepts to fine-grained details, which serves as a structured reasoning scaffold and improves the modeling efficiency. We further investigate the potential of utilizing continuous latent states to improve the reasoning capability of generative models. Finally, we address external guidance by aligning generative models with user-defined goals and environmental feedback. For discrete biological sequences, we introduce DRAKES, a direct reward backpropagation algorithm that optimizes oracle feedback while enforcing distributional constraints to prevent reward hacking. For diffusion language models, we propose SPG, a policy gradient algorithm designed to improve reasoning capabilities via verifiable outcome rewards. To address the challenge of likelihood estimation in this context, we derive a sandwiched objective that leverages both the evidence lower and upper bounds, enabling stable and effective optimization.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
https://rightsstatements.org/page/InC-EDU/1.0/
Copyright retained by author(s)
Persistent DSpace Link
https://dspace.mit.edu/handle/1721.1/178352
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.