Stiff neural ordinary differential equations
Name
2103.15341.pdf
Description
Accepted version
Size
1.51 MB
Format
Adobe PDF
Checksum (MD5)
fe64c1ff5998753e3cec2522ebd28b7b
Author(s)
Kim, Suyong
Ji, Weiqi
Deng, Sili
Ma, Yingbo
Rackauckas, Christopher
Date Issued
September 2021
Journal
Chaos: An Interdisciplinary Journal of Nonlinear Science
Publisher
AIP Publishing
Citation
Kim, Suyong, Ji, Weiqi, Deng, Sili, Ma, Yingbo and Rackauckas, Christopher. 2021. "Stiff neural ordinary differential equations." Chaos: An Interdisciplinary Journal of Nonlinear Science, 31 (9).
Version
Author's final manuscript
Abstract
Neural Ordinary Differential Equations (ODE) are a promising approach to
learn dynamic models from time-series data in science and engineering
applications. This work aims at learning Neural ODE for stiff systems, which
are usually raised from chemical kinetic modeling in chemical and biological
systems. We first show the challenges of learning neural ODE in the classical
stiff ODE systems of Robertson's problem and propose techniques to mitigate the
challenges associated with scale separations in stiff systems. We then present
successful demonstrations in stiff systems of Robertson's problem and an air
pollution problem. The demonstrations show that the usage of deep networks with
rectified activations, proper scaling of the network outputs as well as loss
functions, and stabilized gradient calculations are the key techniques enabling
the learning of stiff neural ODE. The success of learning stiff neural ODE
opens up possibilities of using neural ODEs in applications with widely varying
time-scales, like chemical dynamics in energy conversion, environmental
engineering, and the life sciences.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1063/5.0060697
