Physics-constrained machine learning strategies for turbulent flows and bubble dynamics

Abstract

Machine learning (ML) has in recent years become a sizzling trend in almost every science and engineering discipline. It enables scientists and engineers to make decisions or draw conclusions directly using information extracted from data, bypassing the necessity to unravel the delicate inner workings of the underlying phenomena. This, however, comes at the expense of having to search through an immense space of potential architectures and parameters for an optimized model that, not only provides the best description to the available data, but also applies to unseen cases. To cope with such difficulties, it is imperative that ample constraints are imposed on the architecture and parameter space, in order to facilitate efficient and generalizable learning. For physical systems, first-principle knowledge makes up a natural set of constraints that should be integrated into the ML system.

Past research efforts have mainly emphasized utilizing physical knowledge for cases where the system states are perfectly defined. For reduced-order settings, this condition is not satisfied and consequently, the existing physical knowledge is often incomplete and/or being compromised in accuracy. As a result, it remains a challenge to effectively leverage such knowledge in the design and implementation of ML frameworks. The objective of this thesis is to address the existing gaps and present physics-constrained ML strategies which work specifically in reduced-order spaces. The first part of this thesis focuses on using ML to improve imperfect reduced-order models, obtained from two common methods: (i) orthogonal subspace projection and (ii) slow-manifold reduction. The first case is particularly important for the modeling of rare extreme events such as those found in geophysical sciences.

In these problems, there is typically little data available to train a data-driven model, while reduced-order models of the full equations also fail to capture the relevant dynamics. We present a modeling strategy that allows the ML and physical model to complement each other. Specifically, physics-based equations are projected to a subspace which contains critical dynamical components associated with rare events and then combined with a data-driven model. In this way, the projected equations assist in modeling these events, which appear less frequently in data streams. On the other hand, observations are often plentiful in other regions of the state space, allowing ML to capture dynamics unaccounted by the projected equations. The effectiveness of this strategy is demonstrated through the prediction and modeling of extreme dissipation events in turbulent fluid flows.

Next we present a strategy for improving slow-manifold reduced-order models, suitable for systems with separated time scales. Our strategy employs ML to model the 'fast variables' of the system in terms of the 'slow variables', which can then be integrated with the equation-based dynamics of the latter to provide a complete description of the system evolution. In this way, we constrain ML to a specific part of the dynamics which cannot be easily derived or expressed analytically. We demonstrate the strategy through the modeling of finite-size (inertial) particle dynamics in generic fluid flows, a problem of critical importance for modeling bubbles in multi-phase flows, aerosols in the atmosphere, and ocean drifters. We first utilize training data obtained from the classical Maxey-Riley equation of motion and secondly via high-fidelity multi-phase direct numerical simulations. In both cases, we show that the kinematics, i.e.

the relationship between position (slow variable) and velocity (fast variable), can be effectively learned from limited trajectories and directly utilized to model interactions with complex, turbulent flows. We carefully study transferability of the ML models into different fluid flows and different parameters. To deal the problem of limited data the ML approach is complemented with a data-augmentation technique that enforces physical symmetries of the problem such as isotropicity of the particle dynamics. In the second part of the thesis, we consider a complementary problem related to the parameterization of the unmodeled variables of a reduced-order model with respect to the modeled/dynamically-resolved reduced-order states. This problem is particularly meaningful when explicit dynamical modeling of the target variables is prohibitive.

We present a formulation of this problem in the context of atmospheric modeling, where the spatially small-scaled features are much harder to measure/model than the corresponding large scales due to the high intrinsic dimensionality and the lack of predictability resulting from instabilities. To address these issues, we introduce the Stochastic Machine-Learning (SMaL) parameterization framework that decomposes the times-series for the small scales into a deterministic (predictable) and a stochastic (unpredictable) component. The deterministic component is directly captured with ML in terms of the large scale time-series. On the other hand, the local-in-time statistics of the stochastic components are estimated and learned with separate models. We then construct a non-stationary Gaussian process which enables efficiently drawing an ensemble of small-scaled trajectories that are consistent with the large scales.

The SMaL framework is illustrated on a realistic application, where the small-scaled vorticity fields are parameterized in terms of the large-scaled vorticity and temperature data from reanalysis over Europe. We show that the small-scaled random samples exhibit realistic characteristics in terms of the spatial spectrum, single-point probability density functions, and temporal spectral content.