Decoding of Facial Strains via Conformable Piezoelectric Interfaces and Three-Dimensional Digital Image Correlation

Abstract

Devices that facilitate nonverbal communication typically require high computational loads or have rigid and bulky form factors that are unsuitable for use on the face or on other curvilinear body surfaces. This work reports the design and pilot testing of an integrated system for decoding facial strains and for predicting facial kinematics. The system consists of massmanufacturable, conformable piezoelectric thin films for strain mapping; multiphysics modelling for analysing the nonlinear mechanical interactions between the conformable device and the epidermis; and three-dimensional digital image correlation (3D-DIC) for reconstructing softtissue surfaces under dynamic deformations as well as for informing device design and placement. Most biomedical sensor designs lack an in-depth study of the target soft tissue before the design and fabrication of the sensor meant to couple to that tissue. This work demonstrates the use of 3D-DIC as a method for in-depth biokinematic study of the target region upon which a sensor with mechanically-active functional material, such as piezoelectrics, will be placed. Similar to how chemical assays of a body part would be conducted before designing medication for disorders of that body part, so too does 3D-DIC allow for the mechanical study of biological soft tissue before designing the mechanically-active functional materials on mechanically-adaptive substrates that are meant to intimately integrate with that soft tissue. Thus, the 3D-DIC allowed for the design of a conformable piezoelectric device that mimics the properties of skin and that can interpret and distinguish facial strains in real-time and with low computational load, i.e. with reduced data streaming bandwidth. Finally, pilot studies on healthy individuals and on individuals with amyotrophic lateral sclerosis show that these conformable piezoelectric devices, coupled with algorithms for the real-time detection and classification of distinct skin-deformation signatures, enable the reliable decoding of facial movements. The integrated system could be adapted for use in clinical settings as a nonverbal communication technology or for use in the monitoring of neuromuscular conditions.