Towards a Perceptual Loss: Using a Neural Network Codec Approximation as a Loss for Generative Audio Models
Name
2019-07-ACMM-Ishwarya-Loss.pdf
Description
Accepted version
Size
3.51 MB
Format
Unknown
Checksum (MD5)
f6ffa2434b0179bb792c3791dbf87d5a
Author(s)
Ananthabhotla, Ishwarya
Ewert, Sebastian
Paradiso, Joseph A
Date Issued
2019
Journal
MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia
Publisher
Association for Computing Machinery (ACM)
Citation
Ananthabhotla, Ishwarya, Ewert, Sebastian and Paradiso, Joseph A. 2019. "Towards a Perceptual Loss: Using a Neural Network Codec Approximation as a Loss for Generative Audio Models." MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia.
Version
Author's final manuscript
Abstract
© 2019 Association for Computing Machinery. Generative audio models based on neural networks have led to considerable improvements across fields including speech enhancement, source separation, and text-to-speech synthesis. These systems are typically trained in a supervised fashion using simple element-wise ℓ1 or ℓ2 losses. However, because they do not capture properties of the human auditory system, such losses encourage modelling perceptually meaningless aspects of the output, wasting capacity and limiting performance. Additionally, while adversarial models have been employed to encourage outputs that are statistically indistinguishable from ground truth and have resulted in improvements in this regard, such losses do not need to explicitly model perception as their task; furthermore, training adversarial networks remains an unstable and slow process. In this work, we investigate an idea fundamentally rooted in psychoacoustics. We train a neural network to emulate an MP3 codec as a differentiable function. Feeding the output of a generative model through this MP3 function, we remove signal components that are perceptually irrelevant before computing a loss. To further stabilize gradient propagation, we employ intermediate layer outputs to define our loss, as found useful in image domain methods. Our experiments using an autoencoding task show an improvement over standard losses in listening tests, indicating the potential of psychoacoustically motivated models for audio generation.
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Program in Media Arts and Sciences (Massachusetts Institute of Technology)
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3343031.3351148
