Slip Through the Chat: Subtle Injection of False Information in LLM Chatbot Conversations Increases False Memory Formation
Name
3708359.3712112.pdf
Size
2.78 MB
Format
Adobe PDF
Checksum (MD5)
4326853d6c68b6b9f3fc0a710ae3cd4a
Author(s)
Pataranutaporn, Pat
Archiwaranguprok, Chayapatr
Chan, Samantha
Loftus, Elizabeth
Maes, Pattie
Date Issued
March 24, 2025
Publisher
ACM|30th International Conference on Intelligent User Interfaces
Citation
Pat Pataranutaporn, Chayapatr Archiwaranguprok, Samantha W. T. Chan, Elizabeth Loftus, and Pattie Maes. 2025. Slip Through the Chat: Subtle Injection of False Information in LLM Chatbot Conversations Increases False Memory Formation. In Proceedings of the 30th International Conference on Intelligent User Interfaces (IUI '25). Association for Computing Machinery, New York, NY, USA, 1297–1313.
Version
Final published version
Abstract
This study examines the potential for malicious generative chatbots to induce false memories by injecting subtle misinformation during user interactions. An experiment involving 180 participants explored five intervention conditions following the presentation of an article: (1) no intervention, (2) reading an honest or (3) misleading article summary, (4) discussing the article with an honest or (5) misleading chatbot. Results revealed that while the misleading summary condition increased false memory occurrence, misleading chatbot interactions led to significantly higher rates of false recollection. These findings highlight the emerging risks associated with conversational AI as it becomes more prevalent. The paper concludes by discussing implications and proposing future research directions to address this concerning phenomenon.
Description
IUI ’25, Cagliari, Italy
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3708359.3712112
