Barrier Function to Skin Elasticity in Talking Head
Name
12559_2024_Article_10344.pdf
Size
884.31 KB
Format
Adobe PDF
Checksum (MD5)
aaaf3ade8797c2fdc4e7a9347f535708
Author(s)
Chaturvedi, Iti
Pandelea, Vlad
Cambria, Erik
Welsch, Roy
Datta, Bithin
Date Issued
August 24, 2024
Journal
Cognitive Computation
Publisher
Springer US
Citation
Chaturvedi, I., Pandelea, V., Cambria, E. et al. Barrier Function to Skin Elasticity in Talking Head. Cogn Comput (2024).
Version
Final published version
Abstract
In this paper, we target the problem of generating facial expressions from a piece of audio. This is challenging since both audio and video have inherent characteristics that are distinct from the other. Some words may have identical lip movements, and speech impediments may prevent lip-reading in some individuals. Previous approaches to generating such a talking head suffered from stiff expressions. This is because they focused only on lip movements and the facial landmarks did not contain the information flow from the audio. Hence, in this work, we employ spatio-temporal independent component analysis to accurately sync the audio with the corresponding face video. Proper word formation also requires control over the face muscles that can be captured using a barrier function. We first validated the approach on the diffusion of salt water in coastal areas using a synthetic finite element simulation. Next, we applied it to 3D facial expressions in toddlers for which training data is difficult to capture. Prior knowledge in the form of rules is specified using Fuzzy logic, and multi-objective optimization is used to collectively learn a set of rules. We observed significantly higher F-measure on three real-world problems.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s12559-024-10344-7
