Multimodal AI for Human Sensing and Interaction
Name
3706599.3706651.pdf
Size
378.6 KB
Format
Adobe PDF
Checksum (MD5)
b75238f533acef9352ff4193d56adf3e
Author(s)
Liang, Paul Pu
Ahuja, Karan
Luo, Yiyue
Date Issued
April 25, 2025
Publisher
ACM|Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
Citation
Paul Pu Liang, Karan Ahuja, and Yiyue Luo. 2025. Multimodal AI for Human Sensing and Interaction. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25). Association for Computing Machinery, New York, NY, USA, Article 821, 1–4.
Version
Final published version
Abstract
A significant body of HCI research today focuses on applying AI to sense, learn, and interact with humans through a wide range of wearable and ubiquitous sensors. These methods typically involve learning features from multimodal sensory data using AI methods. To aid HCI researchers who want to apply AI to their sensing problems, this course will cover the fundamental challenges and approaches in multimodal AI for human sensing and interaction. It is planned for 3 parts, one given by each organizer. The first covers the foundations of multimodal AI, studying how AI systems can represent, combine, and learn information from many interconnected sensory inputs. The second part discusses the practice of multimodal AI for human sensing, covering the latest methods for cross-modal learning across diverse sensors, human-centered application domains, and real-world concerns around their usage. The final part covers the hardware, fabrication, and data collection challenges that must be tackled to deploy these multimodal AI systems in the real world. By the end of this course, attendees should understand the fundamental principles and challenges of multimodal AI, identify the right AI approaches for their problems, prototype basic hardware systems for efficient and robust sensing, be aware of real-world concerns around ethics, interpretability, and privacy, and appreciate the range of human-centered applications enabled by multimodal AI and sensing.
Description
CHI EA ’25, Yokohama, Japan
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3706599.3706651
