Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection
Name
mti-09-00022.pdf
Size
2.02 MB
Format
Adobe PDF
Checksum (MD5)
71957159cbb28f2faa17408838b95f14
Author(s)
Watson, Eleanor
Nguyen, Minh
Pan, Sarah
Zhang, Shujun
Date Issued
March 3, 2025
Journal
Multimodal Technologies and Interactions
Publisher
Multidisciplinary Digital Publishing Institute
Citation
Watson, E.; Nguyen, M.; Pan, S.; Zhang, S. Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection. Multimodal Technol. Interact. 2025, 9, 22.
Version
Final published version
Abstract
Value alignment for AI is not “one-size-fits-all”: even polite and friendly models can still fail to represent individual user contexts and preferences, and local cultural norms. This paper presents a modular workflow for personal fine-tuning, synthesizing four core components from our previous research: (1) robust vectorization of user values and preferences, (2) a binary choice user interface (UI) approach to capturing those preferences with minimal cognitive load, (3) contrastive activation methods for steering large language models (LLMs) via difference vectors, and (4) knowledge graph integration for more auditable and structured alignment. Our approach—descended from past research on “Towards an End-to-End Personal Fine-Tuning Framework”—demonstrates how these elements can be combined to create personalized, context-rich alignment solutions. We report on user studies for the forced-choice UI, describe an experimental pipeline for deriving “control vectors”, and propose a “moral graph” method for bridging symbolic and vector-based alignment. Our findings suggest that multi-pronged personalization can significantly reduce user annotation fatigue, improve alignment fidelity, and allow for more flexible, interpretable AI behaviors.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3390/mti9030022
