betaFeedback

WorksPhysics and Astronomy

Data-driven nonlinear turbulent flow scaling with Buckingham Pi variables

Abstract

Nonlinear machine learning for turbulent flows can exhibit robust performance even outside the range of training data. This is achieved when machine-learning models can accommodate scale-invariant characteristics of turbulent flow structures. This study presents a data-driven approach to reveal scale-invariant vortical structures across Reynolds numbers that provide insights for supporting nonlinear machine-learning-based studies of turbulent flows. To uncover conditions for which nonlinear models are likely to perform well, we use a Buckingham Pi-based sparse nonlinear scaling to find the influence of the Pi groups on the turbulent flow data. We consider nonlinear scalings of the invariants of the velocity gradient tensor for an example of three-dimensional decaying isotropic turbulence. The present scaling not only enables the identification of vortical structures that are interpolatory and extrapolatory for the given flow field data but also captures non-equilibrium effects of the energy cascade. As a demonstration, the present findings are applied to machine-learning-based super-resolution analysis of three-dimensional isotropic turbulence. We show that machine-learning models reconstruct vortical structures well in the interpolatory space with reduced performance in the extrapolatory space revealed by the nonlinearly scaled invariants. The present approach enables us to depart from labeling turbulent flow data with a single parameter of Reynolds number and comprehensively examine the flow field to support training and testing of nonlinear machine-learning techniques.

Introduction

Trained fluid mechanicians can identify similarities in vortical structures for a variety of turbulent flows. Even if there are scale or rotational differences, we can visually extract similar structures due to their geometrical features across spatial and temporal scales. Analogously, recent machine-learning models have capitalized on such structural similarities to achieve reliable performance for the given training data. However, it is also known that machine-learning models often achieve satisfactory performance even in the cases of unseen Reynolds numbers. Examples of models performing successfully beyond the training data include, turbulence modeling [1], state estimation [2], and super-resolution [3]. This suggests that machine-learning models are able to incorporate scale-invariant properties of the flows while optimizing the output to meet their objectives.

These observations imply that nonlinear machine-learning models are capturing data characteristics in a more holistic manner. Traditionally, interpolation and extrapolation of models have been associated with a particular variable or parameter [4, 5], as illustrated in Figure 1(a). In this example, test data outside of the training data is captured by the temporal variable tt. In the case of more complex data set, such as turbulent flows, describing whether a certain type of vortical structure appears in training data or not requires additional considerations, beyond a single parameter such as the Reynolds number.

Image not extracted

Figure 1. Concept of interpolation and extrapolation. (a) A one-dimensional example. (b) Evolution of three-dimensional decaying isotropic turbulence over Reλ(t)Re_\lambda(t).

Given this motivation, we revisit the idea of interpolation and extrapolation in the context of turbulent flow structures. Here, we consider the capturing vortical structures that have certain similarities across training and test data to be “interpolation” and their structural features to be “seen.” On the other hand, capturing structures in the test data that do not share similarities with those in the training data are referred to as “extrapolation” and their structures as “unseen.” With machine-learning models known to perform well for approximating the nonlinear relationship between the input and output from a large collection of data, the presence of seen (or common) features can provide robust performance under untrained flow situations [3, 1]. The current study presents a data-driven scaling approach to reveal seen/unseen structures of turbulent flow data in machine learning.

To assess rotational and shear similarities between small- and large-length scales in turbulent flows, we examine the flow field data in terms of the invariants of the velocity gradient tensor. In the present analysis, the scaled invariants are found through sparse nonlinear regression using non-dimensional parameters from the Buckingham Pi theorem [6]. The present approach offers the optimal nonlinear scalings of the invariants, uncovering scale-invariant vortical structures in a turbulent flow field. Analyzing the data distribution of the present scaled invariants reveals what types of flow structures are seen and unseen. Furthermore, the findings from this study provide guidance in the choice of machine-learning functions to offer robustness for scale-invariant vortical structures. The present paper is organized as follows. The proposed Buckingham-Pi sparse nonlinear scaling of the invariants is introduced in section 2. We demonstrate the current approach for three-dimensional isotropic turbulence in section 3. Concluding remarks are provided in section 4.

Methods

Machine-learning models for turbulent flow structures are known to remain accurate beyond the coverage of training data. This is the case especially when the nonlinear machine-learning models have scale and rotational invariances embedded in their formulations. To explain this extended validity of machine-learning models, this study aims to uncover nonlinear scalings that capture the similarities in turbulent vortical structures across a range of Reynolds numbers. The central hypothesis of this study is that the existence of such scalings enables nonlinear machine-learning techniques to effectively perform across different flow fields beyond the range of Reynolds numbers provided in the training data. With these identified scalings, turbulent flow structures that linearly and nonlinearly span across a range of scales may be considered seen beyond the training data due to their structural similarities.

To examine turbulent flows, we consider the invariants of the velocity gradient tensor A=(∇u)TA = \left(\nabla u\right)^T such that the observations are independent of the frame of reference [7]. These invariants are P=trace⁡(A)P = \operatorname{trace}(A), Q=12[P2−trace⁡(A2)]Q = \frac{1}{2}[P^2 - \operatorname{trace}(A^2)], and R=det⁡(A)R = \det(A). For the present study, we consider incompressible turbulent flows, which makes P=∇⋅u=0P = \nabla\cdot u = 0. The remaining two invariants of QQ and RR characterize the local rotation and shear, respectively [8]. According to these invariants, the flow can experience vortex compression (Q>0,R<0Q > 0, R < 0), vortex stretching (Q>0,R>0Q > 0, R > 0), biaxial strain (Q<0,R<0Q < 0, R < 0), and axial strain (Q<0,R>0Q < 0, R > 0) [9]. These invariants will be nonlinearly scaled with non-dimensional Pi groups using a data-driven approach.

In this study, we consider three-dimensional incompressible decaying isotropic turbulence. The flow field data is obtained from direct numerical simulation using 64364^3 grid points with a Taylor microscale-based Reynolds number of 0.85≤Reλ≤2520.85 \le Re_\lambda\le252 [10, 11], satisfying kmax⁡η≥1k_{\max}\eta\ge1, where kmax⁡k_{\max} is the maximum resolvable wavenumber of the grid and η\eta is the Kolmogorov length scale, to resolve all important scales of motion. The present simulation is initialized with random velocity fields of Gaussian profiles that satisfy incompressibility and the Kolmogorov spectra for kinetic energy. The flow snapshots are curated after the flow reaches the decaying regime at which we set time tt to zero. Note that a large ReλRe_\lambda at the early stage of decay is due to a small dissipation coefficient, which strongly depends on the flow [12, 13]. We present representative flows with their corresponding invariants on the QQ–RR plane over time in Figure 2. The Reynolds number decreases as vortical structures evolve as their characteristic size increases. Even at a large time when Reλ=4.18Re_\lambda= 4.18, large-scale structures are still observed while the flow is under decay. During this process, invariants QQ and RR decrease in magnitude while the probability density functions of these invariants remain geometrically similar. This suggests that there is some level of scale invariance in the distributions of the turbulent vortical structures over the QQ–RR plane. That is, the decaying isotropic turbulence holds similar rotational and shear structures whose sizes vary over the Reynolds number.

Image not extracted

Figure 2. Example flow snapshots with QQ–RR distributions of three-dimensional decaying turbulence at (a) Reλ=214Re_\lambda= 214, (b) 14.614.6, and (c) 4.184.18. Each distribution is colored by density. Turbulent vortices are visualized with (a) Q=10Q = 10, (b) 0.30.3, and (c) 0.020.02.

Linear scaling for the QQ–RR distributions based on the kinematic viscosity ν\nu and energy dissipation rate ϵ\epsilon such as Q/(ϵ/ν)Q/(\epsilon/\nu) and R/(ϵ/ν)3/2R/(\epsilon/\nu)^{3/2} do not completely collapse the distributions, given their long tails especially at high Reynolds number, as presented in Figure 3. This seems to be caused by the wide range of Reynolds numbers and strong unsteadiness contained in the present decaying turbulence data, implying that Kolmogorov’s similarity hypotheses do not hold in an instantaneous manner. For example, skewness SRS_R of the linearly-scaled distribution for RR in Figure 2 is 2.742.74, 0.2990.299, and 0.1860.186 for Reλ=214Re_\lambda= 214, 14.614.6 and 4.184.18, respectively. These observations suggest that nonlinear scaling needs to be considered to accommodate non-equilibrium effects. Identifying such nonlinear scalings of QQ and RR distributions with data-driven techniques can also reveal how nonlinear machine learning extracts seen (or common) vortical structures that share similarities with structures outside of the training data sets. Moreover, unseen structures can be captured by uncovering the invariant space that do not overlap for the scaled data.

Linearly-scaled $Q$–$R$ distributions

Figure 3. Linearly-scaled QQ–RR distributions based on the kinematic viscosity ν\nu and energy dissipation rate ϵ\epsilon. A zoom-in view for Reλ=14.6Re_{\lambda}=14.6 and 4.184.18 is also shown.

Let us consider scaling the invariants QQ and RR in a nonlinear manner using non-dimensional Pi groups from the Buckingham-Pi theorem (1914), which distills a number of dimensional parameters into a smaller number of dimensionless groups [14, 15]. We assume that the scaling can be obtained through superposing appropriate polynomials of the Pi variables. Denoting these invariants as ϕ\phi (either QQ and RR), the candidate polynomials are assumed to have the form of

θk(x,t)=Πim(t)Πjn(t)ϕ(x,t),where i,j=1,2,…,m, n∈Z,k=1,…,nL.(1)\theta_k(x,t) = \Pi_i^m(t)\Pi_j^n(t)\phi(x,t), \quad\text{where } i,j = 1,2,\ldots,m,\ n \in\mathbb{Z},\quad k = 1,\ldots,n_L. \tag*{(1)}

Here, nLn_L is the number of library-basis functions. Given these library candidates θk\theta_k, we can express the scaled invariants ϕ∗(x,t)\phi^*(x,t) as

ϕ∗(x,t)=∑k=1nLakθk(x,t)(2)\phi^*(x,t) = \sum_{k=1}^{n_L} a_k\theta_k(x,t) \tag*{(2)}

with a=(a1,a2,…,anL)T∈RnL\boldsymbol{a} = (a_1,a_2,\ldots,a_{n_L})^T \in\mathbb{R}^{n_L}. Spatiotemporal discretization of this equation yields Φ∗=Θa\boldsymbol{\Phi}^* = \boldsymbol{\Theta}\boldsymbol{a}, where

Φ∗=[ϕ∗(t1)ϕ∗(t2)⋮ϕ∗(tnt)]∈Rnxnt,Θ=[θ1(t1)θ2(t1)⋯θnL(t1)θ1(t2)θ2(t2)⋯θnL(t2)⋮⋮⋱⋮θ1(tnt)θ2(tnt)⋯θnL(tnt)]∈Rnxnt×nL(3)\Phi^{*} = \begin{bmatrix} \phi^{*}(t_{1}) \\ \phi^{*}(t_{2}) \\ \vdots\\ \phi^{*}(t_{n_{t}}) \end{bmatrix} \in\mathbb{R}^{n_{x}n_{t}}, \qquad \Theta= \begin{bmatrix} \theta_{1}(t_{1}) & \theta_{2}(t_{1}) & \cdots& \theta_{n_{L}}(t_{1}) \\ \theta_{1}(t_{2}) & \theta_{2}(t_{2}) & \cdots& \theta_{n_{L}}(t_{2}) \\ \vdots& \vdots& \ddots& \vdots\\ \theta_{1}(t_{n_{t}}) & \theta_{2}(t_{n_{t}}) & \cdots& \theta_{n_{L}}(t_{n_{t}}) \end{bmatrix} \in\mathbb{R}^{n_{x}n_{t}\times n_{L}} \tag*{(3)}

with ϕ(t)=(ϕ(x1,t),…,ϕ(xnx,t))T∈Rnx\phi(t)=(\phi(x_{1},t),\ldots,\phi(x_{n_{x}},t))^{T}\in\mathbb{R}^{n_{x}} and θk(t)=(θk(x1,t),…,θk(xnx,t))T∈Rnx\theta_{k}(t)=(\theta_{k}(x_{1},t),\ldots,\theta_{k}(x_{n_{x}},t))^{T}\in\mathbb{R}^{n_{x}} being the scaled invariant and the library candidate, respectively, discretized in space. Here, the invariants are spatiotemporally stacked into a tall vector. The coefficients aa are determined through the sequential threshold least squares method [16], promoting sparsity of the coefficient matrix in a computationally efficient manner [17]. Since excessive sparsity promotion leads to a high regression error, the sparsity coefficient in the threshold least square method needs to be carefully tuned [18].

For the present decaying turbulence, the Taylor length scale λ\lambda can be expressed as the function of the characteristic velocity uu (the square root of the spatially averaged kinetic energy), the kinematic viscosity ν\nu, the computational domain size LL, and viscous dissipation ϵ\epsilon such that λ=f(u,ν,L,ϵ)\lambda=f(u,\nu,L,\epsilon). Through the use of Buckingham Pi theorem, we can find three Pi variables, which namely are Π1=uλ/ν\Pi_{1}=u\lambda/\nu, Π2=uL/ν\Pi_{2}=uL/\nu, and Π3=ϵν/u4\Pi_{3}=\epsilon\nu/u^{4}. The first Pi variable Π1\Pi_{1} is the Taylor Reynolds number ReλRe_{\lambda}. In the present study, we take these Pi variables as a function of time. Given these Pi variables, we can construct the library candidates for equation (1) as first and second-order polynomials of the Pi variables such that

{Π1, Π2, Π3, Π1−1, Π2−1, Π3−1, Π12, Π22, Π32, Π1−1Π2, Π2−1Π3, Π3−1Π1, …, ΠimΠjn, …},(4)\{\Pi_{1},\ \Pi_{2},\ \Pi_{3},\ \Pi_{1}^{-1},\ \Pi_{2}^{-1},\ \Pi_{3}^{-1},\ \Pi_{1}^{2},\ \Pi_{2}^{2},\ \Pi_{3}^{2},\ \Pi_{1}^{-1}\Pi_{2},\ \Pi_{2}^{-1}\Pi_{3},\ \Pi_{3}^{-1}\Pi_{1},\ \ldots,\ \Pi_{i}^{m}\Pi_{j}^{n},\ \ldots\}, \tag*{(4)}

where ∣m∣+∣n∣≤2|m|+|n|\leq2.

With the Buckingham Pi-based library matrix Θ\Theta, we seek coefficients aa by maximizing the similarity of data distributions of the invariants over space and time. Here, we utilize the Kullback–Leibler (KL) divergence [19] to assess the difference between a probability distribution f1f_{1} and the reference probability distribution f2f_{2}. For two probability (data) distributions, the KL divergence is defined as

D(f1∥f2)≡∫−∞∞f1(ϕ∗)log⁡f2(ϕ∗)f1(ϕ∗) dϕ∗.(5)D(f_{1}\Vert f_{2})\equiv\int_{-\infty}^{\infty}f_{1}(\phi^{*})\log\frac{f_{2}(\phi^{*})}{f_{1}(\phi^{*})}\,\mathrm{d}\phi^{*}. \tag*{(5)}

The minimization of the KL divergence finds the optimal coefficients a∗a^{*} for maximum similarity of the scaled invariant distributions, which yields an optimization problem of a∗=arg min⁡a[D(f1∥f2)]a^{*}=\operatorname*{arg\,min}_{a}[D(f_{1}\Vert f_{2})]. Below, the data distribution from a snapshot at high ReλRe_{\lambda} is used as the reference f2f_{2}.

Results

We apply the present nonlinear scaling analysis to three-dimensional decaying isotropic turbulence over 0.85≤Reλ≤2520.85\leq Re_{\lambda}\leq252. The present data is comprised of 800 snapshots over time and 64364^{3} grid points in a tri-periodic cubic domain. Here, we aim to (1) uncover the nonlinear influence of characteristic variables on the evolution of the invariants QQ and RR, (2) identify common (seen) vortical flow features through the scaled invariants, and (3) provide guidance on proper training and formulation of machine-learning models.

Based on sparse regression, we find the scaling factors for QQ and RR to be

Q∗=(5.76Π1+4.17Π2−2−3.59Π2Π3)Q,(6)Q^{*} = (5.76\Pi_{1} + 4.17\Pi_{2}^{-2} - 3.59\Pi_{2}\Pi_{3})Q, \tag*{(6)}
R∗=(5.53Π1−0.826Π2−1+4.42Π2−2−3.44Π2Π3)R.(7)R^{*} = (5.53\Pi_{1} - 0.826\Pi_{2}^{-1} + 4.42\Pi_{2}^{-2} - 3.44\Pi_{2}\Pi_{3})R. \tag*{(7)}

While isotropic turbulence is complex, the nonlinearly scaled invariants turn out to be surprisingly compact in their expressions. For both scaled invariants, we have Π1\Pi_{1} which is the Taylor Reynolds number ReλRe_{\lambda}. This reflects the decaying nature, corresponding to the fact that the data spread on the QQ–RR plane shrinks with decreasing Taylor Reynolds number over time, as shown in Figure 4(a) and (Figure 4b). The present scaling also identifies the influence of Π2=uL/ν\Pi_{2} = uL/\nu, the box-size-based Reynolds number, revealing that the computational domain size influences the turbulent flow, especially as time advances and vortical structures become comparable in size to the computational domain. This is evident from the inverse and quadratic inverse nature of the scalings. The size of the periodic box affects not only the large-scale vortical motion over the domain but also the energy dissipation of turbulence [9].

Image not extracted

Figure 4. The data-driven Buckingham Pi scaling for three-dimensional decaying turbulence. The time series of the identified non-dimensional variables as a function of (a) time and (b) the Taylor Reynolds number. (c) Scaled Q∗Q^{*} and R∗R^{*} invariants with their probability density functions. A zoom-in view is also shown as a subfigure.

Furthermore, the present analysis uncovers the importance of Π2Π3=uL/ν⋅ϵν/u4=ϵL/u3\Pi_{2}\Pi_{3} = uL/\nu\cdot\epsilon\nu/u^{4} = \epsilon L/u^{3}, which is the ratio between the instantaneous energy dissipation rate determined by small length scales and the cascading energy flux from the system-size vortices. Thus, Π2Π3\Pi_{2}\Pi_{3} plays a similar role to the dissipation coefficient Cϵ≡ϵl/u3C_{\epsilon} \equiv\epsilon l/u^{3}, quantifying the degree of non-equilibrium [20], i.e., the violation of Kolmogorov’s similarity. Although Π2Π3\Pi_{2}\Pi_{3} is constant when the energy flux of large-scale vortices and energy dissipation rate are in balance, that is not the case for the present decaying turbulence [21]. The present Π2Π3\Pi_{2}\Pi_{3} accounts for the time delay of energy-cascade process between energy flux and dissipation due to strong nonlinearities in decaying turbulence. This is achieved by correcting non-equilibrium effects in a nonlinear manner. Note that the present formulation provides similar scaling expression in a range of similar Reynolds numbers even with different initial flow fields since sparse regression captures how the turbulent flow decays over time.

Next, the identified scalings are applied to the original QQ and RR data distributions, as shown in Figure 4(c) along with their probability density functions. The identified factors in equations 6 and 7 yield the optimal overlap of Q∗Q^{*} and R∗R^{*} over space and time. The scaled data distribution for the high Reynolds number flow spreads over a larger area than that for the low Reynolds number, especially for Q∗>0Q^{*} > 0. This implies high occurrence of vortex stretching and compression at high Reynolds number.

Let us focus on the vortical structures for overlapping and non-overlapping regions of the data over the Q∗Q^{*}–R∗R^{*} plane, as illustrated in Figure 5. For Q∗>1Q^{*} > 1, similar shapes of vortical elements can be observed across a range of length scales. These identifiable vortical structures are assessed as ‘interpolatory’ (common) with the present scaling approach. The isosurfaces of Q∗>3Q^{*} > 3 are also visualized in Figure 5. These strong rotational elements that barely appear in low Reynolds number flows are ‘extrapolatory’ structures. Moreover, such ‘extrapolatory’ vortical structures against the low-ReRe flow field can be seen in the portion of R∗<−1.2R^{*} < -1.2 and R∗>1.2R^{*} > 1.2. The region of Q<0Q < 0 and R<0R < 0 corresponds to biaxial strain, while that of Q>0Q > 0 and R>0R > 0 reflects vortex stretching [9]. Hence, the scaled data distribution suggests that these structures caused by strong biaxial strain and vortex stretching correspond to ‘extrapolation’ for the present turbulence data. While not shown here, the portion of R∗>−0.5R^{*} > -0.5 and −3.5<Q∗<0.5-3.5 < Q^{*} < 0.5 also includes extrapolatory vortical structures. The scaled QQ–RR data distribution gives insights into turbulent flows, in addition to the identified scaling.

Image not extracted

Figure 5. Interpolatory and extrapolatory vortical structures in three-dimensional decaying isotropic turbulence.

The present Buckingham-Pi-based sparse nonlinear scalings can identify interpolatory and extrapolatory vortical structures in isotropic turbulence. Nonlinear machine-learning models are likely to capture the characteristics and behavior of what we refer to as the interpolatory structures even in untrained Reynolds number cases. This explains why well-trained machine-learning models may perform well even for testing data. However, we caution that when the test data includes extrapolatory structures, machine-learning models are no longer guaranteed to be valid. With regard to these points, classifying interpolatory and extrapolatory structures solely by ReλRe_\lambda is not encouraged for assessing nonlinear machine-learning models.

Based on the insights from the scaled invariants above, let us consider machine-learning-based super-resolution reconstruction of turbulent flows [22]. Super resolution reconstructs the high-resolution flow field qHRq_{\mathrm{HR}} from its low-resolution data qLRq_{\mathrm{LR}} with a reconstruction model F\mathcal{F} through qHR=F(qLR)q_{\mathrm{HR}}=\mathcal{F}(q_{\mathrm{LR}}). Recently developed machine-learning-based super-resolution analysis captures the nonlinear relationship between small (unresolved) and large-scale (resolved) structures. This study considers the ability of machine-learning-based reconstruction to recognize and reconstruct common turbulent flow structures for a variety of flow field snapshots, even outside of the training data.

For super-resolution reconstruction of turbulent flows, a machine-learning model should be carefully constructed to account for a range of spatial length scales while enforcing rotational and translational invariance of vortical structures. To satisfy these properties, we use the hybrid downsampled skip-connection/multi-scale (DSC/MS) model [23, 24]. The DSC/MS model based on convolutional neural network [25] is composed of three main functions: namely, (A) up/downsampling operations, (B) skip connection, and (C) multi-scale filters. The up/downsampling provides robustness against rotation and translation of vortical structures. The skip connection allows communication between the input low-resolution data and the intermediate output of the DSC model, which is crucial in learning a step-by-step internal process towards the high-resolution output from the low-resolution input while expanding the dimension of the flow field snapshot [26]. The multi-scale filters apply filtering with a number of different sizes of them (e.g., three here) in parallel to capture a broad range of scales in turbulent flows. Further details on the present neural network model are in Fukami et al. [23]. We will discuss which function inside the present model contributes to gaining robustness for scale invariance later.

Here, we consider two cases of training and testing for super-resolution reconstruction of turbulent flows: (1) a model trained with a low-ReλRe_\lambda data set and tested with a high-ReλRe_\lambda data set (low-ReλRe_\lambda training); (2) a model trained with a high-ReλRe_\lambda data set and tested with a low-ReλRe_\lambda data set (high-ReλRe_\lambda training). We expect that the low-ReλRe_\lambda training cannot cover the non-overlapping portion of the scaled invariants, which corresponds to extrapolation. The high-ReλRe_\lambda training, which covers a wide portion of the scaled invariants, may amount to interpolation in terms of the turbulent flow structures. For the present analysis with the two training scenarios, the threshold ReλRe_\lambda between the low- and high-ReλRe_\lambda training cases is set to Reλ=15Re_\lambda= 15. The velocity vector is used as data attributes q\boldsymbol{q} with F\mathcal{F} reconstructing the high-resolution flow field on 64364^3 grids from the low-resolution data on 434^3 grid.

The reconstructed flow fields are shown in Figure 6(a). The vortical flows are visualized using QQ-criteria and colored by the third invariant RR computed from the reconstructed velocities. For the low-ReλRe_\lambda training case, the model reconstructs the vortical structures for the training ReλRe_\lambda with an L2L_2 error of approximately 0.1. In contrast, the visualized QQ variable at the high-ReλRe_\lambda exhibits significant level of error rendering the reconstruction grossly incorrect. Such high errors can be explained using the scaled variables in Figure 6(b) due to the flow features residing over the non-overlapping region on the present plane. The high error is also observed around the bottom left of Figure 6(b). This extrapolation region cannot be reconstructed with the low-ReλRe_\lambda training because these structures are barely seen in the low-ReλRe_\lambda data sets, as presented in Figure 5.

Image not extracted

Figure 6. (a) Super-resolution reconstruction of three-dimensional decaying turbulence. The reconstructed flow fields are visualized with the QQ-criteria, colored by RR. The values underneath each figure represent the L2L_2 error norm. The gray and red boxes respectively highlight snapshots for training and testing ReλRe_\lambda regime. (b) Scaled Q∗Q^* and R∗R^* at test ReλRe_\lambda’s for low- and high-ReλRe_\lambda training cases, colored by the spatial L2L_2 reconstruction error.

Next, let us consider the high-ReλRe_\lambda training case. The reconstructed flow from the overlapping case (interpolation) is in agreement with the reference data. In contrast to the extrapolatory low-ReλRe_\lambda training, reasonable reconstruction can still be achieved at Reλ=4.18Re_\lambda= 4.18. This suggests that the high-ReλRe_\lambda training data holds insights into a wider range of vortical structures that also appear in the low ReλRe_\lambda regime, which is confirmed from the scaled Q∗Q^* and R∗R^* data. It is worth pointing out that the scaled invariants at the low ReλRe_\lambda include some non-overlapping portion (scale-variant structures) in the region of Q∗<0Q^* < 0 and R∗<0R^* < 0. This implies that the model trained with only the high-ReλRe_\lambda regime cannot cover vortical structures of Q∗<0Q^* < 0 and R∗<0R^* < 0. For better reconstruction over this regime, we need to include training data that covers Q∗<0Q^* < 0 and R∗<0R^* < 0. Such an observation provides guidance on how the vortical flow data should be prepared to enable reliable reconstruction.

We further examine the nonlinearly scaled invariants to assess which inner functions of the machine-learning models contribute to robustness for scale-invariant regression. We here consider six models using the aforementioned functions A, B, and C (Figure 7(a)):

The three functions used in the DSC/MS super-resolution model and scaled $Q^*$-$R^*$ data for six models

Figure 7. (a) The three functions used in the DSC/MS super-resolution model. (b) The scaled Q∗Q^*-R∗R^* data for six models. The two-dimensional sections of the reconstructed streamwise velocity are also shown with velocity errors.

  1. the original DSC/MS model (functions A, B, and C),

  1. the original model without multi-scale filters (functions A and B),

  1. the original model without the skip connection (functions A and C),

  1. the original model with up/downsampling only (function A),

  1. the original model with multi-scale filters only (function C),

  1. a regular convolutional neural network (without any of the functions above).

These six models consider all possible combinations that can be constructed with the DSC/MS model. We perform the low-ReλRe_\lambda training for all of these six cases. The corresponding error distributions are presented in Figure 7(b). For model (ii) which removes the multi-filter function from the baseline model, the error behavior on the scaled data is similar to that of the original model. A similar observation is seen for models (iii) and (iv) which have no multi-filter functions. Since models (ii-iv) include function A, up/downsampling operations are crucial for robust super-resolution reconstruction of turbulent flows.

This can be further confirmed with model (v) which is comprised only of the multi-scale filters (function C). The reconstruction error is significantly higher without the up/downsampling operations. Including dimension compression and expansion plays an important role in obtaining robustness for scale-invariant characteristics, which agrees with a number of studies on CNNs for scale invariance characteristics in image science [27, 28]. In contrast, multi-scale filters have secondary importance in reconstructing the flow (model (v)) as evident from its comparison to the results from CNN (model (vi)). While the regular CNN without functions A, B, and C returns pixelized flow fields, model (v) achieves qualitative reconstruction. The use of skip connections is also important for successful training to address issues related to the convergence of neural network weights [26]. These findings suggest that robust turbulent flow reconstruction can be achieved by selecting machine-learning functions based on the implication of scaled invariants.

The current approaches to identify important model functions that accommodate scale invariance could be combined with emerging methods to investigate the role of networks in capturing flow physics, such as the sensitivity analysis [29, 30] and Fourier analysis of the CNN filters [31]. Along with these techniques, model effectiveness and accuracy should be carefully examined in terms of its construction and the richness of training data for machine-learning applications of turbulent flows.

Concluding remarks

For nonlinear machine-learning models for turbulent flows, there is generally not a single parameter that can reveal whether such models are performing an interpolation or extrapolation. This is due to the turbulent flow data containing similarities in flow structures across a range of spatiotemporal scales. These properties contribute to machine-learning models being accurate beyond the coverage of training data in some cases. To shed light on the validity of machine-learning models, we nonlinearly scaled the invariants of the velocity gradient tensor QQ and RR with non-dimensional parameters using a Buckingham Pi theorem-based sparse regression, which maximizes the similarity of invariant data distributions over space and time across Reynolds numbers. As a canonical turbulent flow example, we considered three-dimensional decaying isotropic turbulence for 0.85≤Reλ≤2520.85 \le Re_{\lambda} \le252. The present approach found nonlinear scalings that express the influence of the decaying nature of the present flow, domain size of the simulation, and non-equilibrium effects of energy cascade on the invariants QQ and RR. With the scaled invariants Q∗Q^{*} and R∗R^{*}, we were able to determine that the training data lacked flow structures associated with strong biaxial strain and vortex stretching.

We further analyzed which types of machine-learning functions contribute to gaining robustness for scale-invariant vortical structures within the context of super-resolution reconstruction. We found that fluid flow reconstruction can be achieved for data of the overlapping portion on the scaled Q∗Q^{*}–R∗R^{*} plane even under untrained ReλRe_{\lambda} with an appropriate construction of machine-learning models. The present findings suggest that transfer learning could be effective for training nonlinear machine-learning models of turbulent flow across Reynolds number provided that extrapolatory structures do not alter the physics significantly [32, 2]. Including fractional exponents for Pi variables would likely enhance the generalizability of the present method for discovering additional nonlinear scaling in turbulent flows. Incorporating nonlinearly scaled invariants into the training process could also support the generalizability of the models. While we can consider the scaled invariants as the input and output of machine-learning models, they could also be incorporated into the loss function of machine-learning optimization. The present procedure to examine nonlinear scalings of turbulent flow structures provides guidance on how to develop robust machine-learning models and compile the necessary training data, enabling us to depart from naïve training and being unaware of the validity of these complex models.

Acknowledgements

KF thanks the support from the UCLA-Amazon Science Hub for Humanity and Artificial Intelligence. KT acknowledges the generous support from the US Air Force Office of Scientific Research (FA9550-21-1-0178) and the US Department of Defense Vannevar Bush Faculty Fellowship (N00014-22-1-2798). DNS data was generously shared by Dr. M. Gopalakrishnan Meena.

Declaration of interests

The authors report no conflict of interest.

References

  1. [1]Y. Guan, A. Chattopadhyay, A. Subel, and P. Hassanzadeh. Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning. J. Comput. Phys., 458:111090, 2022.DOI
  2. [2]L. Guastoni, A. Güemes, A. Ianiro, S. Discetti, P. Schlatter, H. Azizpour, and R. Vinuesa. Convolutional-network models to predict wall-bounded turbulence from wall quantities. J. Fluid Mech., 928:A27, 2021.arxiv.org/abs/2006.12483
  3. [3]H. Kim, J. Kim, S. Won, and C. Lee. Unsupervised deep learning for super-resolution reconstruction of turbulence. J. Fluid Mech., 910:A29, 2021.DOI
  4. [4]P. Domingos. A few useful things to know about machine learning. Commun. ACM, 55(10):78–87, 2012.DOI
  5. [5]G. Marcus. Deep learning: A critical appraisal. arXiv:1801.00631, 2018.arxiv.org/abs/1801.00631
  6. [6]E. Buckingham. On physically similar systems; illustrations of the use of dimensional equations. Phys. Rev., 4(4):345–376, 1914.DOI
  7. [7]M. S. Chong, A. E. Perry, and B. J Cantwell. A general classification of three-dimensional flow fields. Phys. Fluids A: Fluid Dynamics, 2(5):765–777, 1990.
  8. [8]A. Ooi, J. Martin, J. Soria, and M. S. Chong. A study of the evolution and characteristics of the invariants of the velocity-gradient tensor in isotropic turbulence. J. Fluid Mech., 381:141–174, 1999.DOI
  9. [9]P. A. Davidson. Turbulence: an introduction for scientists and engineers. Oxford university press, 2015.
  10. [10]S. G Chumakov. A priori study of subgrid-scale flux of a passive scalar in isotropic homogeneous turbulence. Phys. Rev. E, 78(3):036313, 2008.pubmed.ncbi.nlm.nih.gov/18851149
  11. [11]M. Gopalakrishnan Meena and K. Taira. Identifying vortical network connectors for turbulent flow modification. J. Fluid Mech., 915:A10, 2021.DOI
  12. [12]S. Goto and J. C. Vassilicos. The dissipation rate coefficient of turbulence is not universal and depends on the internal stagnation point structure. Phys. Fluids, 21(3), 2009.DOI
  13. [13]S. Goto and J. C. Vassilicos. Energy dissipation and flux laws for unsteady turbulence. Phys. Lett. A, 379(16–17):1144–1148, 2015.DOI
  14. [14]X. Xie, A. Samaei, J. Guo, W. K. Liu, and Z. Gan. Data-driven discovery of dimensionless numbers and governing laws from scarce measurements. Nat. Commun., 13(7562), 2022.DOI
  15. [15]J. Bakarji, J. Callaham, S. L. Brunton, and J. N. Kutz. Dimensionally consistent learning with Buckingham Pi. Nat. Comput. Sci., 2:834–844, 2022.arxiv.org/abs/2202.04643
  16. [16]S. L. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. U.S.A., 113(15):3932–3937, 2016.DOI
  17. [17]K. Fukami, T. Murata, K. Zhang, and K. Fukagata. Sparse identification of nonlinear dynamics with low-dimensionalized flow representations. J. Fluid Mech., 926:A10, 2021.DOI
  18. [18]E. Kaiser, J. N. Kutz, and S. L. Brunton. Sparse identification of nonlinear dynamics for model predictive control in the low-data limit. Proc. Roy. Soc. A, 474(2219):20180335, 2018.
  19. [19]S. Kullback and R. A. Leibler. On information and sufficiency. Ann. Math. Stat., 22(1):79–86, 1951.DOI
  20. [20]J. C. Vassilicos. Dissipation in turbulent flows. Annu. Rev. Fluid Mech., 47:95–114, 2015.DOI
  21. [21]S. Goto and J. C. Vassilicos. Local equilibrium hypothesis and Taylor’s dissipation law. Fluid Dyn. Res., 48(2):021402, 2016.DOI
  22. [22]K. Fukami, K. Fukagata, and K. Taira. Super-resolution analysis via machine learning: a survey for fluid flows. Theor. Comput. Fluid Dyn., 37:421–444, 2023.DOI
  23. [23]K. Fukami, K. Fukagata, and K. Taira. Super-resolution reconstruction of turbulent flows with machine learning. J. Fluid Mech., 870:106–120, 2019.DOI
  24. [24]K. Fukami, K. Fukagata, and K. Taira. Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows. J. Fluid Mech., 909:A9, 2021.DOI
  25. [25]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86(11):2278–2324, 1998.
  26. [26]K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.arxiv.org/abs/1512.03385
  27. [27]K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In IEEE 12th ICCV, pages 2146–2153, 2009.DOI
  28. [28]N. Van Noord and E. Postma. Learning scale-variant and scale-invariant features for deep image classification. Pattern Recognit., 61:583–592, 2017.arxiv.org/abs/1602.01255
  29. [29]S. Lee and D. You. Analysis of a convolutional neural network for predicting unsteady volume wake flow fields. Phys. Fluids, 33(035152), 2021.DOI
  30. [30]E. Jagodinski, X. Zhu, and S. Verma. Inverse identification of dynamically important regions in turbulent flows using three-dimensional convolutional neural networks. Phys. Rev. Fluids, 8(9):094605, 2023.DOI
  31. [31]A. Subel, Y. Guan, A. Chattopadhyay, and P. Hassanzadeh. Explaining the physics of transfer learning in data-driven turbulence modeling. PNAS nexus, 2(3):pgad015, 2023.DOI
  32. [32]M. Inubushi and S. Goto. Transfer learning for nonlinear dynamics and its application to fluid turbulence. Phys. Rev. E, 102(4):043301, 2020.DOI

Paper details

vlnc:W_zquhfh2r
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents