User Profiling Based on Nonlinguistic Audio Data
Name
User Profiling Based on Nonlinguistic Audio Data.pdf
Size
3.45 MB
Format
Adobe PDF
Checksum (MD5)
2969927ba8983189cebc8f61a58cd45f
Author(s)
Shen, Jiaxing
Cao, Jiannong
Lederman, Oren
Tang, Shaojie
Pentland, Alex
Date Issued
September 7, 2021
Publisher
ACM Transactions on Information Systems
Citation
Shen, J., Cao, J., Lederman, O., Tang, S., & Pentland, A. S. (2021). User Profiling Based on Nonlinguistic Audio Data. ACM Transactions on Information Systems (TOIS), 40(1), 1-23.
Abstract
User profiling refers to inferring people’s attributes of interest (AoIs) like gender and occupation, which enables various applications ranging from personalized services to collective analyses. Massive nonlinguistic audio data brings a novel opportunity for user profiling due to the prevalence of studying spontaneous face-to-face communication. Nonlinguistic audio is coarse-grained audio data without linguistic content. It is collected due to privacy concerns in private situations like doctor-patient dialogues. The opportunity facilitates optimized organizational management and personalized healthcare, especially for chronic diseases. In this article, we are the first to build a user profiling system to infer gender and personality based on nonlinguistic audio. Instead of linguistic or acoustic features that are unable to extract, we focus on conversational features that could reflect AoIs. We firstly develop an adaptive voice activity detection algorithm that could address individual differences in voice and false-positive voice activities caused by people nearby. Secondly, we propose a gender-assisted multi-task learning method to combat dynamics in human behavior by integrating gender differences and the correlation of personality traits. According to the experimental evaluation of 100 people in 273 meetings, we achieved 0.759 and 0.652 in F1-score for gender identification and personality recognition, respectively.
MIT Department
MIT Connection Science (Research institute)
Terms of Use
Attribution-NonCommercial-ShareAlike 3.0 United States
Persistent DSpace Link
