betaFeedback

WorksPhysics and Astronomy

Invariance-embedded Machine Learning Sub-grid-scale Stress Models for Meso-scale Hurricane Boundary Layer Flow Simulation I: Model Development and a priori\textit{a priori} Studies

Abstract

This study develops invariance-embedded machine learning sub-grid-scale (SGS) stress models admitting turbulence kinetic energy (TKE) backscatter towards more accurate large eddy simulation (LES) of meso-scale turbulent hurricane boundary layer flows. The new machine learning SGS model consists of two parts: a classification model used to distinguish regions with either strong energy cascade or energy backscatter from those with mild TKE transfer and a regression model used to calculate SGS stresses in regions with strong TKE transfer. To ease model implementation in computational fluid dynamics (CFD) solvers, the Smagorinsky model with a signed coefficient CsC_s, where a positive value indicates energy cascade while a negative one indicates energy backscatter, is employed as the carrier of the machine learning model. To improve its robustness and generality, both physical invariance and geometric invariance features of turbulent flows are embedded into the model input for classification and regression, and the signed Smagorinsky model coefficient is used as the output of the regression model. Different machine-learning methods and input setups have been used to test the classification model's performance. The F1-scores, which measure balanced precision and recall of a model, of the classification models with physical and geometric invariance embedded can be improved by about 17%17\% over those without considering geometric invariance. Regression models based on ensemble neural networks have demonstrated superior performance in predicting the signed Smagorinsky model coefficient, exceeding that of the dynamic Smagorinsky model in a priori\textit{a priori} tests.

Invariance-embedded Machine Learning Sub-grid-scale Stress Models for Meso-scale Hurricane Boundary Layer Flow Simulation I: Model Development and a priori Studies

Abstract

Introduction

Recent improvements in computing technology have significantly boosted the accuracy of numerical weather models, especially for predicting tropical cyclones (TCs), thus allowing for greater detail and precision in forecasting extreme weather events. Direct numerical simulation (DNS) and large-eddy simulation (LES) are conventional methods for simulating turbulent flows. However, these methods are not computationally feasible for numerical simulations of TCs, which have numerous energy-containing vortices in the eyewall region. For example, traditional sub-grid-scale (SGS) stress models in different applications[1, 2, 3] are mainly designed to model turbulence of a typical length scale around meters, designated as “micro-scale” phenomena within the domain of geophysical flows. Therefore, their computational cost is prohibitive for a macro-scale weather pattern, such as hurricanes, spanning hundreds of kilometers. Meso-scale non-hydrostatic atmospheric flow simulations[4, 5] can significantly reduce the computational cost of TCs. However, judicious numerical dissipation, such as localized artificial viscosity[6], must be developed to add an adequate amount of artificial diffusion to stabilize numerical simulations. In many applications[7, 8, 9, 10], traditional SGS models, which only consider forward-directed turbulence kinetic energy (TKE) cascade, were used to provide numerical dissipation, which, unfortunately, can introduce an overabundance of numerical dissipation on meso-scale hurricane simulations. Acknowledging the importance of energy backscatter in atmospheric and oceanic models has been demonstrated crucial for realistic meso-scale simulations[11, 12, 13].

In SGS closure modeling, machine learning (ML) has introduced innovative approaches to overcome certain turbulence energy transfer challenges. ML’s performance in managing multi-dimensional data and using complex non-linear correlations has made it an enticing toolkit[14, 15, 16, 17]. It offers the prospect of uncovering hidden insights from data and delivering superior SGS closures alongside novel perspectives on SGS physics. Guan et al.[18] found that using convolutional neural networks (CNNs) with transfer learning can create accurate, stable, and widely applicable LES models admitting energy backscatter for 2D decaying turbulence. They achieved this through a priori and a posteriori analyses to develop non-local, data-driven SGS models. The development of data-driven SGS closures for canonical geophysical flows, including 2D and quasigeostrophic turbulence as well as broader atmospheric and oceanic circulations, has yielded promising results[19, 20, 21, 22]. Notably, deep neural networks (NNs) trained on high-fidelity DNS data have facilitated the emergence of stable and precise LES with data-driven SGS closures[23, 24].

We notice that using simulation data directly does not show reasonable estimations of energy backscatter, even from a priori tests of the model performance[25]. The use of frame independence showed better performance at turbulence physics prediction based on the flow information in its neighborhood[26]. Decomposing the velocity gradient into its symmetric and anti-symmetric components and embedding invariances[27], which are derived from the high fidelity data, also showed a better performance in adding domain knowledge to machine learning models.

SGS stress models in turbulence modeling can be classified into functional[28] and structural[29] models. Functional models primarily use an eddy viscosity approach to approximate the effects of unresolved scales by adding effective viscosity to enhance kinetic energy dissipation. This method is straightforward and effective in reproducing the overall energy transfer between resolved and subgrid scales. However, it often fails to accurately predict the SGS term, leading to inaccuracies in local flow structures[30]. Structural models, on the other hand, focus on accurately capturing the local SGS stresses and dynamics by reconstructing the smaller scales. While they offer higher accuracy and preserve detailed flow structures, they can be computationally demanding and prone to instability due to incorrect backscatter predictions[7, 31]. The idea of energy backscatter captures the essence of energy dynamics within TCs, influencing wind speed distribution and the shape of eyewalls and rain bands. In fluid dynamics, energy backscatter means a non-linear and often inverse transfer of energy, which can greatly affect turbulence and, in turn, the weather patterns they cause[32, 33, 34, 35, 36]. To mitigate the adverse effect of inaccurate backscatter modeling, hybrid approaches that combine eddy viscosity components[31] with structural models can be employed to retain accuracy while damping excessive backscatter. Additionally, adaptive filters or limiting functions can be used to reduce backscatter[37, 38] selectively, and introducing hyper-viscosity[39, 40] can stabilize the simulation by enhancing dissipation at smaller scales. In a recent study[41], organized kinetic energy backscatter was observed using radar measurements in the hurricane boundary layer. It demonstrates that meso-scale SGS modeling requires non-equilibrium flow effects for accurate dynamics capture.

In this work, we introduce a novel and interpretable framework that integrates machine learning techniques with physical and geometric invariances to enhance the accuracy of SGS models admitting energy backscatter for meso-scale simulations. A key strength of this approach lies in its explainability. The framework improves predictive performance by embedding domain-specific knowledge into the raw data using tensor invariants and singular values of local flow features. It provides a clear connection to the underlying physics of unresolved turbulence. Unlike traditional “black-box” ML models, this framework explicitly incorporates physical constraints and rotational and reflective invariances, ensuring consistency with fundamental turbulence principles. Using these invariances, the model respects the underlying flow structures, enhancing its interpretability and reliability. The research evaluates various NN configurations, including direct feed-forward networks and ensemble-based approaches, demonstrating their ability to stabilize and improve predictions under highly variable flow conditions. The framework improves the prediction of both forward energy cascade and energy backscatter in mesoscale atmospheric conditions resembling hurricanes, as demonstrated in a priori tests.

The remainder of the paper is organized as follows. The main machine learning framework, including classification and regression models, is briefly introduced in § 1.3. Section III describes data preparation processes that create suitable inputs for the ML models and briefly introduces the ML configurations employed for classification and regression. Here, we highlight the systematic incorporation of physical and geometric invariances, which ensures that the models align with the physical laws governing turbulent flows. Section IV presents and compares the results of the classification and regression ML models. Specifically, Section IV A highlights the ability of classification models to identify critical regions with strong backscatter and forward energy cascade, while Section IV B demonstrates the regression models’ capability of accurately predicting the signed coefficient CsC_s in the Smagorinsky model. The performance comparison between the new ML model and dynamic Smagorinsky model in an a priori sense is presented in Section IV C. Finally, § 1.6 summarizes the findings and discusses future works.

Framework for Machine Learning SGS Stress Modeling

This section gives an overview of the invariance-embedded ML SGS stress modeling framework. The Smagorinsky[42] model is used as the baseline SGS stress model in this study. It reads

νt=CsΔ2∣S‾ij∗∣,(1)\nu_t = C_s \Delta^2 \left|\overline{S}^{*}_{ij}\right|, \tag*{(1)}

where νt\nu_t is the turbulence eddy viscosity, CsC_s is the model constant, Δ\Delta is the filter size, and S‾ij∗\overline{S}^{*}_{ij} is the deviatoric strain rate tensor with its magnitude ∣S‾ij∗∣\left|\overline{S}^{*}_{ij}\right| defined as 2S‾ij∗S‾ij∗\sqrt{2\overline{S}^{*}_{ij}\overline{S}^{*}_{ij}}. Note that in the classic Smagorinsky model, the coefficient CsC_s is required to be positive as the eddy viscosity νt\nu_t is an analogy of the molecular kinematic viscosity. However, many atmospheric and oceanic modeling/simulation studies, such as those reviewed in § 1.2, have emphasized the importance of energy backscatter in meso-scale SGS modeling. Therefore, in this study, instead of treating νt\nu_t as the eddy viscosity, we redefine it as the turbulence momentum diffusion/anti-diffusion coefficient and assume that it follows the format of the original Smagorinsky model, i.e., Eq. (1). As a result, νt\nu_t can be positive or negative, and CsC_s in the Smagorinsky model becomes a signed coefficient. Although anti-diffusion can cause numerical instability, it has been reported by Guan et al.[18] that accurate modeling of turbulence energy backscatter can ensure stable a posteriori LES of turbulence simulations.

As will be reported in Section IV C, the dynamic Smagorinsky model (DSM)[43] can reasonably predict the variation trend of the signed Smagorinsky coefficient CsC_s. However, it is very challenging for DSM to predict the true values of CsC_s accurately. This violates the accuracy requirement posed on energy backscatter modeling in Ref. 18. Therefore, we develop an invariance-embedded ML SGS stress modeling framework in this work, which can accurately predict the signed Smagorinsky coefficient CsC_s. Specifically, this framework consists of a classification model that can distinguish CsC_s with large and small absolute values with a threshold of 0.001 and a regression model that can map localized flow field to signed CsC_s in a point-to-point sense, thus facilitating practical ML SGS model implementation for LES in a posteriori tests.

As shown in Figure 1, physical and geometric invariances are extracted from the raw data to embed domain-specific knowledge into the ML model. Specifically in the pre-processing stage, physical invariances, such as tensor invariants from the combination of the strain rate tensor and vorticity tensor, are extracted from the raw outputs of high-fidelity simulation data[44] of a hurricane-like vortex (see Sect. III C 1). Geometric invariances are then encapsulated via singular value decomposition (SVD) of the normalized local physical invariance fields, producing corresponding singular value fields with essential structural information about the flow (see Sect. III C 2). As the last step of the data pre-processing procedure, the singular value fields are normalized, which transforms to ensure consistency and comparability across the dataset.

Framework for enhanced sub-grid-scale stress modeling with invariance-embedded machine learning classification and regression models

Figure 1. An illustration of the framework for enhanced sub-grid-scale stress modeling with invariance-embedded machine learning classification and regression models.

The normalized singular value fields are input into an ML classifier, such as the feedforward NN for binary classification. This classifier determines whether the local signed SGS model coefficient (CsC_s) should be categorized as small or large based on the learning from the physical and geometric properties of the data. The output from this classification model subsequently informs a regression model based on a feedforward NN structure. This regression model fine-tunes the signed SGS model coefficient (CsC_s) predictions, which are critical for accurate TKE transfer modeling. In the final phase, the outputs from the regression models are combined using an ensemble approach, producing a reliable prediction of the SGS model constants. Next, we explain the technical details of data preparation and ML model creation.

Data Preparation and Machine Learning Model Creation

The datasets used in this study originated from the high-fidelity simulations by Ren et al.[44], featuring a flow field resolution of approximately 62 m. The LES was conducted with the Weather Research and Forecasting Model (WRF)[45], facilitating an accurate presentation of small-scale turbulence patterns crucial to our study. These simulations accounted for varying sea surface temperatures (SSTs), i.e., 26°C, 27°C, 28°C, and 29°C, with the understanding that SSTs above 26°C play a pivotal role in the intensification of hurricanes. Such temperature variations significantly influence a hurricane’s strength and structure, directly impacting potential damages along coastal areas. For an in-depth understanding of these simulations, readers are referred to the work of Ren et al.[44] and Rotunno et al.[46].

Figure 2 presents the comparison of wind speed fields from simulations with coarse (1666.67 m) and fine (61.72 m) grid resolutions at the SST of 29°C, highlighting the differences in accurately representing turbulent characteristics at different grid spacings. The larger grid size of about 1.67 km (Figure 2(a)) depicts the major fluctuations in wind speed, giving a broad picture view of the hurricane’s structure without details of the turbulent flow. A narrower grid spacing of 61.72 m (Figure 2(b)) reveals the effect of turbulent eddies, which can cause local high wind speed that is not captured by the coarse-resolution simulation.

Comparison of instantaneous horizontal wind speed plots for the hurricane center at different grid resolutions

Figure 2. Comparison of instantaneous horizontal wind speed (m/sm/s) plots for the hurricane center in a horizontal plane at the height of 303 m with different grid resolutions.

This study will use the detailed patterns from the high-resolution simulation to figure out important turbulence factors, like eddy viscosity and Smagorinsky model coefficient (CsC_s), the basis for our machine/deep learning model. In turn, this model is meant to make storm predictions more accurate. The procedures and methodologies used to refine this data for NN training are further explained in later sub-sections.

Filtering Fine-resolution LES data

Since only velocity gradient features are used in turbulence modeling with the Smagorinsky model, velocity components in three dimensions (u,v,w)(u, v, w) were smoothed with a Gaussian filter. In the Gaussian filter, the weight assigned to each point in a one-dimensional series is expressed as:

w^±i(Δ,n)=exp⁡(−(iΔ/n)22σ2),\hat{w}_{\pm i}(\Delta,n)=\exp\left(\frac{-(i\Delta/n)^2}{2\sigma^2}\right),

normalized to ensure the sum of all weights equals one:

w±i(Δ,n)=w^±i(Δ,n)∑w^±i(Δ,n).w_{\pm i}(\Delta,n)=\frac{\hat{w}_{\pm i}(\Delta,n)}{\sum\hat{w}_{\pm i}(\Delta,n)}.

Here, nn is the number of points in the filter window, σ\sigma is the standard deviation of the Gaussian distribution, which controls the spread of the Gaussian curve, and Δ\Delta represents the distance between consecutive points in the data series and sets the spacing in the filter application. In this study, Δ\Delta is set as 1 km.

Figure 3(a) shows a hurricane simulation’s horizontal wind speed field at an SST of 29°C. After applying a Gaussian filter with a standard deviation (σ)(\sigma) of 2 (Figure 3(b)), smaller-scale features are reduced, highlighting the primary structures such as the hurricane’s eye and eyewall. Increasing σ\sigma to 20 (Figure 3(c)) smooths the field further, emphasizing larger, more organized patterns while diminishing smaller turbulence.

Figure 3. Horizontal wind speed (m/s)(m/s) field and the corresponding fields after using a Δ=1\Delta= 1 km Gaussian filter with different standard deviations at the 303 m vertical level.

Figure 4 presents the kinetic energy spectra from the simulation data and those after filtering, illustrating energy distribution across scales. The unfiltered spectrum (blue line) follows a −5/3-5/3 slope, consistent with Kolmogorov’s[47] inertial subrange theory. Post-filtering with different $\sigma values reduces energy at smaller scales, highlighting the filter’s dampening effect, particularly pronounced at higher σ\sigma values[48]. All turbulence features presented in the following (sub)sections are calculated from the original and filtered flow data.

Kinetic energy spectra after using 1 km Gaussian filter at 303 m vertical level

Figure 4. Kinetic energy spectra after using 1 km Gaussian filter at 303 m vertical level.

Smagorinsky Model Coefficient Evaluation

The production of TKE in turbulent flows can be quantitatively linked to the interaction between the Reynolds stresses and the mean strain rate, a relationship captured by the definition of turbulence production (Pdef\mathcal{P}^{\mathrm{def}}):

Pdef=−(uiuj‾−u‾iu‾j)∂u‾i∂xj=τijS‾ij.(2)\mathcal{P}^{\mathrm{def}}=-\left(\overline{u_i u_j}-\overline{u}_i\overline{u}_j\right)\frac{\partial\overline{u}_i}{\partial x_j}=\tau_{ij}\overline{S}_{ij}. \tag*{(2)}

Herein, the operator (⋅‾)(\overline{\cdot}) represents a general filtering or averaging operation, uiu_i is the flow velocity, τij\tau_{ij} is the Reynolds stress tensor, and S‾ij\overline{S}_{ij} is the strain rate tensor of the filtered or averaged flow field. Eq. (2) connects the Reynolds stress tensor with the mean strain rate tensor, which shows how fast TKE is produced due to the mean velocity gradients.

Alternatively, the production of TKE can be modeled through the concept of eddy viscosity (νT\nu_T), a construct within the Boussinesq hypothesis that allows for the approximation of the Reynolds stresses in terms of the mean strain rate[50]. This approximation, as given by Eq. (3), assumes a proportionality between the turbulent stress and the mean strain rate, transferring energy from the larger to smaller scales:

P=(2νTS‾ij∗+τkk3δij)S‾ij=2νTS‾ij∗S‾ij+τkk3S‾ii.(3)\mathcal{P} = \left(2\nu_T\overline{S}_{ij}^{*}+\frac{\tau_{kk}}{3}\delta_{ij}\right)\overline{S}_{ij}=2\nu_T\overline{S}_{ij}^{*}\overline{S}_{ij}+\frac{\tau_{kk}}{3}\overline{S}_{ii}. \tag*{(3)}

Herein, S‾ij∗=12(∂u‾i∂xj+∂u‾j∂xi−23∂u‾k∂xkδij)\overline{S}_{ij}^{*}=\frac{1}{2}\left(\frac{\partial\overline{u}_i}{\partial x_j}+\frac{\partial\overline{u}_j}{\partial x_i}-\frac{2}{3}\frac{\partial\overline{u}_k}{\partial x_k}\delta_{ij}\right) is the deviatoric strain rate tensor and the Smagorinsky eddy viscosity model νT=CsΔ2∣S‾ij∗∣\nu_T=C_s\Delta^2\left|\overline{S}_{ij}^{*}\right|. Note that for a strictly incompressible flow, we have S‾ij∗=S‾ij\overline{S}_{ij}^{*}=\overline{S}_{ij} as the divergence of the filtered or averaged flow velocity field ∂u‾k/∂xk\partial\overline{u}_k/\partial x_k is zero when the partial differentiation operator and the filtering/averaging operator are commutable. By combined Eq. (1)–(3), the Smagorinsky model coefficient CsC_s can be calculated from high-fidelity simulation data and the associated filtered data as

Cs=τ‾ijS‾ij−τkk3S‾ii2Δ2∣S‾ij∗∣S‾ij∗S‾ij.(4)C_s=\frac{\overline{\tau}_{ij}\overline{S}_{ij}-\frac{\tau_{kk}}{3}\overline{S}_{ii}}{2\Delta^2\left|\overline{S}_{ij}^{*}\right|\overline{S}_{ij}^{*}\overline{S}_{ij}}. \tag*{(4)}

We mention that in the original Smagorinsky model Eq. (1), the model coefficient CsC_s is considered a positive one and a constant value over the whole field. The theoretical value was calculated as 0.027 by Lily et al. 51 Based on the WRF documentation, CsC_s is a constant of 0.252=0.06250.25^2=0.0625. However, in practical applications, CsC_s can assume both positive and negative values. We presented the CsC_s distributions at positions of y=0y=0, 10, and 15 km in Figure 5. From Figure 5(b), the large variability of the values of Smagorinsky model coefficient CsC_s can be observed. Note that the yypositions are chosen based on Figure 5(a), in which the hurricane eye and eye-wall flow structures exhibit large spatial variations.

Figure 5. CsC_s values at different positions of yy in the xx-direction at approximately 303 m vertical level.

Data Preprocessing and Organization

Embedding Physical Invariance

Accurately depicting turbulent flows in CFD is essential for precise predictions in engineering and environmental contexts. Recent advancements in data-driven turbulence modeling emphasize selecting physically meaningful and invariant input features to enhance generalizability and interpretability[15, 27, 52, 53]. This study employs tensor invariants derived from the average strain rate tensor S\mathbf{S} and the average rotation rate tensor R\mathbf{R} as input features for our NN models. These invariants, listed in Table I, inherently satisfy rotational and reflectional invariances, providing concise and physically consistent descriptions of the local flow conditions.

InvariantFormulaDescription
λ1\lambda_{1}tr⁡(S2)\operatorname{tr}(\mathbf{S}^{2})Trace of the squared strain rate tensor
λ2\lambda_{2}tr⁡(R2)\operatorname{tr}(\mathbf{R}^{2})Trace of the squared rotation rate tensor
λ3\lambda_{3}tr⁡(S3)\operatorname{tr}(\mathbf{S}^{3})Trace of the cubed strain rate tensor
λ4\lambda_{4}tr⁡(R2S)\operatorname{tr}(\mathbf{R}^{2}\mathbf{S})Trace of the product of squared rotation rate tensor and strain rate tensor
λ5\lambda_{5}tr⁡(R2S2)\operatorname{tr}(\mathbf{R}^{2}\mathbf{S}^{2})Trace of the product of squared rotation rate tensor and squared strain rate tensor

Table I. Tensor Invariants Used as Input Features

Incorporating these physical invariances ensures that the predictive capability of NN models remains independent of coordinate systems, a crucial advantage for accurately modeling complex mesoscale hurricane flows. Using invariant input features simplifies the NN’s learning process by explicitly embedding physical invariance, significantly reducing complexity and improving prediction robustness compared to raw velocity fields or TKE[25, 54]. Recent literature underscores the advantage of incorporating such invariances into machine learning frameworks, demonstrating model robustness and predictive accuracy improvements across various turbulent flow scenarios[14, 55]. By embedding these physical invariances directly, our framework enhances the stability and explainability of NN predictions, thereby making it particularly suitable for complex mesoscale hurricane simulations.

Embedding Geometric Invariance

The primary focus of this study lies on the eyewall region of a hurricane, which is renowned for its significant contribution to the storm’s dynamics and strength fluctuations[56]. The annular zone within the storm is recognized as a region with concentrated convective activity and powerful angular momentum[57]. In order to isolate this specific location, we establish a defined area bounded by two concentric circles with radii of 10 and 20 km (see Figure 6(a)). This effectively marks the eyewall. The black dots in this region are those associated with large absolute CsC_s values.

The effect of reflection and rotation by 180° on the actual simulation field

Figure 6. The effect of reflection and rotation by 180° on the actual simulation field.

Note that tensor invariants used in Sect. III C 1 ensure coordinate transformation invariance of pointwise data. It does not present information regarding relative positions (i.e., local pattern) between points. As shown in Figure 6(b) and 6(c), when a flow pattern is mirrored or rotated, it should be characterized as the same flow feature as the original one by the ML model. We, thus, resort to creating ML classification and regression models with flow features, represented by singular values, of a group of points in the flow field to embed geometric invariance into the model input. The group learning concept and its comparison to the pointwise learning concept are shown in Figure 7.

Different dataset preparation for pointwise learning and group learning

Figure 7. Different dataset preparation for (a) pointwise learning and (b) group learning.

Rotation and reflections are a type of orthogonal transformation where the transformation matrix QQ is an orthogonal matrix with a determinant of 1 for rotation and −1-1 for reflection. QQ satisfies the following orthogonality condition:

QTQ=QQT=I.Q^{T}Q = QQ^{T} = I.

Consider the matrix AA, and its reflection or rotation is QAQA, where QQ is the reflection or rotation matrix. To check if the singular values are invariant, consider (QA)T(QA)(QA)^{T}(QA):

(QA)T(QA)=ATQTQA.(QA)^{T}(QA) = A^{T}Q^{T}QA.

Since QQ is orthogonal, we have

(QA)T(QA)=ATIA=ATA.(QA)^{T}(QA) = A^{T}IA = A^{T}A.

This calculation shows that ATAA^{T}A remains unchanged after applying the rotation or reflection transformation. In a geometric sense, both rotation and reflection can be viewed as transformations that alter the orientation or configuration of the matrix AA but do not change the inherent scale lengths measured by the singular values, which are tied to the eigenvalues of ATAA^{T}A. In the context of hurricanes, where wind directions and flow patterns can vary significantly, maintaining invariance to these transformations ensures that the ML model remains robust and generalizable across different storm scenarios.

In group learning, every point and its neighboring points are assembled to form a matrix. Then, SVD is applied to calculate the singular values. This process not only facilitates a reduction in the dimensionality of the input data but also captures the essential characteristics of the data. By focusing on significant singular values, we retain the most influential features of the turbulence data, reducing noise and less relevant information. In this study, we applied SVD to local 3×33 \times3 or 5×55 \times5 matrices extracted around each point of interest within the hurricane’s eyewall.

Machine Learning Methods

This subsection briefly introduces machine learning models, including those for classification and regression, used in this study.

Classification Methods

In the classification task, we focused on predicting whether the absolute value of the local SGS model coefficient (CsC_s) should be categorized as small or large. We employed several machine learning methods, including Logistic Regression (LR), Random Forest (RF), Support Vector Machines (SVM), Gradient Boosting (GB), and Neural Network (NN) to evaluate their effectiveness in classifying the CsC_s values. The detailed study was reported in Ref. 54, and the comparison results for the group learning with geometric invariance embedded are briefly discussed in Appendix § 1.6. Based on the test results, NN was selected for further study due to its robustness and adaptability in capturing complex data patterns across various configurations.

We constructed a feedforward NN using PyTorch[58], a prominent deep-learning library. The network’s design featured a linear, fully connected architecture with several configurations to assess their influence on the model’s predictive capabilities. Key hyperparameters were tuned for optimal performance, such as the number of hidden layers, the number of nodes in each layer, and the activation functions. For the training of NN-based classification models, the standard backpropagation and gradient descent were used to minimize the Binary Cross-Entropy (BCE) Loss[59] defined as:

BCE Loss=−1N∑i=1N[yilog⁡(y^i)+(1−yi)log⁡(1−y^i)],(5)\mathrm{BCE\ Loss}=-\frac{1}{N}\sum_{i=1}^{N}\left[y_i\log(\hat{y}_i)+(1-y_i)\log(1-\hat{y}_i)\right], \tag*{(5)}

where NN is the number of samples, yiy_i is the true label for the ii-th sample (0 or 1), and y^i\hat{y}_i is the predicted probability for the ii-th sample, which is the output of the sigmoid function applied to the model’s output. The model’s output is passed through a sigmoid function to produce probabilities, which are then compared to the true labels using the “BCE Loss” function.

Training NN involves optimizing weights and biases to minimize a cost function using backpropagation and gradient descent[60]. The number of hidden layers, nodes per layer, and learning rate significantly impact the model’s performance. While larger networks with more layers and nodes can model complex data more effectively, they also risk overfitting, where the model performs well on training data but poorly on unseen data[61, 62]. Proper tuning and regularization are essential to balance model complexity and generalization. Performance test results of different NN setups will be presented in Sect. IV.

Regression Methods

The regression task aims to predict the continuous signed values of the Smagorinsky constant (CsC_s) used in turbulence modeling. We employed feedforward NN for this purpose. A dense, fully connected NN was constructed to predict CsC_s values. The NN training involved minimizing the Mean Squared Error (MSE) between the predicted and true CsC_s values defined as:

MSE=1n∑i=1n(yi−y^i)2,(6)\mathrm{MSE} = \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2, \tag*{(6)}

where yiy_i represents the true values, y^i\hat{y}_i represents the predicted values, and nn is the number of data points. We optimized the network using Bayesian regularization[63, 64] to enhance weight estimation. Our recent work has highlighted the effectiveness of invariance-embedded machine learning models for SGS stress predictions, with different optimization strategies yielding varying accuracy and computational efficiency[65].

The regression network’s performance was evaluated through mean squared error metrics across different configurations, including varying the number of neurons and layers. An ensemble approach, combining outputs from multiple networks, was employed to improve prediction accuracy and reduce error variance.

Results and Discussions

Results from Classification Models

Pointwise Learning

After data preparation, we proceeded to the training phase using a feed-forward NN architecture. In pointwise learning, the network was tasked with classifying points based on the magnitude of the Smagorinsky coefficient, CsC_s, using the five tensor invariances shown in Table I as input features. The tensor invariances were divided into positive and negative groups based on a threshold of 0.001 for the magnitude of CsC_s. Labels were assigned accordingly: 1 for positive and 0 for negative samples. We employed a linear model structure, exploring various configurations to assess their influence on the model’s predictive capabilities.

We conducted a series of experiments to understand the effect of network complexity on learning efficiency and generalization. The configurations differed in the number of neurons per layer (10, 50, and 200) and the type of activation function used (ReLU and Tanh) as summarized in Table II.

Model NamesNo. of NeuronsNo. of LayersActivation FunctionNo. of Epochs
NN-1102ReLU50,000
NN-2502ReLU50,000
NN-32002ReLU50,000
NN-4102Tanh50,000
NN-5502Tanh50,000
NN-62002Tanh50,000

Table II. Classification Neural Network Model Configurations

The loss curves in Figure 8 illustrate the training and test loss across epochs for typical configurations, NN-1, NN-3, and NN-4, highlighting the convergence behavior of each model. NN-1 and NN-4 configurations demonstrate a steady reduction in loss over the 50,000 epochs, with the training loss generally lower than the test loss, indicating effective learning but also a possible minor overfitting tendency, as seen in the consistent gap between the two losses. Interestingly, NN-3, despite having significantly more neurons (200 per layer), does not exhibit notably better convergence behavior than simpler models like NN-1 and NN-4. This suggests that increasing model complexity by adding neurons does not necessarily lead to enhanced model performance or improved generalization. The gradual decline in all configurations’ losses suggests that the models effectively learn to classify more accurately as training progresses. The near-convergence of the training and test losses towards the end of training implies a stabilized learning process, where the models balance learning and generalization.

Figure 8. Convergence performance of three typical NNs, i.e., NN-1, NN-3, and NN-4, when pointwise learning is used to train classification NNs.

The quantitative performance of each model configuration is summarized in Table III, presenting key classification metrics. The accuracy reflects the proportion of true results (both true positives and true negatives) among the total number of cases examined. Precision assesses the model’s ability to classify the positive class correctly, while recall measures the model’s capability to identify all positive cases. The F1-Score combines precision and recall into a single metric, balancing learning and generalization. ancing the model’s performance. Here, the formulas for calculating Accuracy, Precision, Recall, and F1-Score are given below:

Model ConfigurationAccuracyPrecisionRecallF1-Score
NN-10.590.600.550.57
NN-20.590.590.610.60
NN-30.600.590.610.60
NN-40.590.600.510.55
NN-50.580.590.520.55
NN-60.580.590.540.56

Table III. NN Model Performance Metrics for Pointwise Training

Accuracy=True Positives (TP)+True Negatives (TN)TP+TN+False Positives (FP)+False Negatives (FN)\mathrm{Accuracy} = \frac{\mathrm{True\ Positives\ (TP)} + \mathrm{True\ Negatives\ (TN)}}{\mathrm{TP} + \mathrm{TN} + \mathrm{False\ Positives\ (FP)} + \mathrm{False\ Negatives\ (FN)}}
Precision=True Positives (TP)True Positives (TP)+False Positives (FP)\mathrm{Precision} = \frac{\mathrm{True\ Positives\ (TP)}}{\mathrm{True\ Positives\ (TP)} + \mathrm{False\ Positives\ (FP)}}
Recall=True Positives (TP)True Positives (TP)+False Negatives (FN)\mathrm{Recall} = \frac{\mathrm{True\ Positives\ (TP)}}{\mathrm{True\ Positives\ (TP)} + \mathrm{False\ Negatives\ (FN)}}
F1 Score=2×Precision×RecallPrecision+Recall\mathrm{F1\ Score} = 2 \times\frac{\mathrm{Precision} \times\mathrm{Recall}}{\mathrm{Precision} + \mathrm{Recall}}

where True Positives (TP): Instances correctly predicted as 1, False Positives (FP): Instances incorrectly predicted as 1, False Negatives (FN): Instances incorrectly predicted as 0, and True Negatives (TN): Instances correctly predicted as 0.

The metrics in tbl:3 demonstrate a relatively uniform accuracy across all configurations, with most models achieving scores around 0.58 to 0.60. This moderate accuracy, paired with precision values close to 0.59 and recall values ranging from 0.51 to 0.61, suggests that while the models can identify positive cases reasonably well, they may struggle with balancing precision and recall consistently across configurations. The F1-Scores, which combine precision and recall, show limited variation across models, indicating that the number of neurons and activation functions did not substantially impact the model’s ability to balance precision and recall. This consistency in F1-Scores and overall performance implies that increasing the model complexity in terms of neurons and activation function did not significantly improve the classification performance of pointwise learning solely with physical invariance embedded into the NN model.

Group Learning without Geometric Invariance

In this section, we developed a custom function to capture local spatial relationships by correlating each data point with its neighbors (Figure 7(b)). Tensor invariants were grouped based on previously described threshold criteria. Balanced sampling ensured representative data coverage and the resulting dataset was randomized and converted to PyTorch tensors for neural network training.

Similar NN from Sub-section IV A 1 were tested with the new dataset under a group learning approach without incorporating geometric invariance. As shown in Figure 9, the training and test loss over epochs reveal the learning progression and provide insights into the model’s generalization capabilities. The models with more neurons show a faster convergence, indicating an enhanced capacity to capture complex data patterns. However, the training and test loss curves remain closely aligned across configurations, suggesting that overfitting was effectively controlled, even with higher neuron counts.

Convergence plots for NN-1 and NN-4

Figure 9. Convergence performance of two typical NNs, i.e., NN-1 and NN-4, when group learning without geometric invariance is used to train classification NNs.

Model ConfigurationAccuracyPrecisionRecallF1-Score
NN-10.600.620.510.56
NN-20.610.600.670.63
NN-30.620.610.670.64
NN-40.600.610.540.57
NN-50.600.610.540.57
NN-60.600.610.550.58

Table IV. NN Model Performance Metrics for Group Learning without Geometric Invariance

Table IV summarizes the model’s performance, which presents accuracy, precision, recall, and F1-score for each configuration. The accuracy across configurations remains relatively uniform, around 0.60 to 0.62, with precision generally around 0.61 to 0.62. Recall values vary more widely, ranging from 0.51 to 0.67, indicating differences in the model’s ability to identify positive cases across configurations. Higher recall values are observed in configurations with more neurons, reflecting their enhanced feature-capturing ability. Overall, the model with 200 neurons and ReLU activation (NN-3) achieved the highest scores across most metrics, suggesting that this configuration was most effective in capturing complex patterns in the data while balancing precision and recall. The consistent patterns across configurations indicate that increasing model complexity, specifically regarding neurons and activation functions, positively impacted classification performance under group learning without geometric invariance.

Group Learning with Geometric Invariance

In contrast to pointwise learning, group learning leverages spatial relationships by considering each data point and its neighboring points. Specifically, we focused on points within a defined annular region of the hurricane boundary layer due to their significance in capturing turbulence dynamics. Neural networks were trained using tensor invariants as inputs to classify the magnitude of the CsC_s. SVD was employed to incorporate geometric invariances into the dataset, ensuring robustness against coordinate variations. Balanced sampling was implemented to maintain unbiased and representative data for effective neural network training.

Figure 10 presents the training and test loss curves over epochs for two representative configurations: NN-1 and NN-4. NN-1 exhibits a steady decline in training and test loss, reflecting a consistent learning process with minimal overfitting. NN-4, while following a similar trajectory, shows a slightly slower convergence, likely due to the smoother gradients provided by the Tanh activation function compared to ReLU.

Convergence performance plots for NN-1 and NN-4

Figure 10. Convergence performance of NNs, i.e., NN-1 and NN-4, when group learning with geometric invariance on a 3×33 \times3 grid (9 points) is used to train classification NNs.

As presented in Table V, the performance metrics reveal the classification results for all configurations. Networks with more neurons, such as NN-3, consistently outperform those with fewer neurons regarding recall (0.75) and F1-Score (0.72). This indicates that larger networks have a greater capacity to model complex patterns within the data, particularly when using ReLU activation. Conversely, models with Tanh activation, such as NN-6, show comparable performance but slightly slower convergence, as observed in the loss curves.

Model ConfigurationAccuracyPrecisionRecallF1-Score
NN-10.700.690.730.71
NN-20.700.690.740.71
NN-30.710.690.750.72
NN-40.700.690.730.71
NN-50.700.690.740.71
NN-60.700.690.740.71

Table V. NN Model Performance Metrics for Group Learning with Geometric Invariance on a 3×33 \times3 grid (9 points)

The study enhanced its data preprocessing method to optimize the classification of turbulence model parameters by incorporating an expanded neighboring context for singular value decomposition. The initial approach focused on a 3×33 \times3 grid (9 points) around a central point, while this enhanced approach considers a larger, 5×55 \times5 grid (25 points). Including a broader neighboring grid likely provides a more comprehensive view of the local turbulence structures. This wider perspective could capture variations in the turbulence characteristics that smaller neighborhoods might miss, allowing for a more nuanced understanding of the local interactions and dependencies. The enhanced feature set obtained via SVD from these 25-point neighborhoods then informs the training of the NN models.

In Figure 11, models trained on a 5×55 \times5 grid exhibit similar trends. The training and test loss curves for NN-1 and NN-4 show a rapid decline in the initial epochs, followed by a plateau, indicating that the expanded spatial context aids in capturing relevant features early in the training process. The consistent alignment between training and test loss curves demonstrates effective generalization.

Convergence performance of NN-1

(a) (a) NN-1

Convergence performance of NN-4

(b) (b) NN-4

Figure 11. Convergence performance of two typical NNs, i.e., NN-1 and NN-4, when group learning with geometric invariance on a 5×55 \times5 grid (25 points) is used to train classification NNs.

Table VI provides the performance metrics for the 5×55 \times5 grid. Across all configurations, accuracy improves to 0.72, and recall increases to 0.76 for models with 200 neurons (NN-3, NN-6). This improvement in recall reflects the model’s enhanced ability to identify true positives, likely attributable to the larger spatial context in the 5×55 \times5 grid, which provides richer information for classification.

Model ConfigurationAccuracyPrecisionRecallF1-Score
NN-10.720.700.750.72
NN-20.720.700.760.73
NN-30.720.700.760.73
NN-40.720.700.750.73
NN-50.720.700.760.73
NN-60.720.700.760.73

Table VI. NN Model Performance Metrics for Group Learning with Geometric Invariance on a 5×55 \times5 grid (25 points)

The results indicate that increasing the neighborhood size for SVD significantly impacts performance metrics, particularly recall and F1-score. The broader spatial context in the 5x5 grid enables the models to capture better local turbulence structures, which are crucial for accurately classifying turbulence dynamics. The improved metrics suggest that spatial relationships in the input features play a more critical role in enhancing classification performance than merely increasing the NN’s complexity. The consistent architecture across configurations highlights the importance of balancing network complexity and activation function to achieve optimal learning. ReLU-based configurations generally converge faster and achieve better classification metrics due to their ability to handle non-linearities and avoid vanishing gradients. Meanwhile, Tanh-based configurations demonstrate a more gradual learning process, which may lead to better generalization under certain conditions.

In summary, transitioning from a 3×33 \times3 grid to a 5×55 \times5 grid demonstrates the importance of incorporating broader spatial contexts in turbulence modeling. The enhanced classification performance across all metrics reinforces the potential of machine learning models, combined with geometric invariance, to provide more accurate and reliable predictions in complex fluid dynamics. These findings underscore the critical role of spatial feature representation in advancing machine-learning-based approaches for turbulence modeling.

Results from Regression Models

This section evaluates the performance of NN models in predicting the signed Smagorinsky coefficient (CsC_s), a critical parameter for turbulence modeling. Both single NNs and ensemble methods are assessed for their effectiveness and reliability in capturing complex patterns in the data.

Single Neural Network Performance

Figure 12 presents a detailed comparison of single NN predictions against the true values of the signed Smagorinsky coefficient. Two representative network configurations are analyzed: one. relatively simple with fewer neurons (n=8,l=2n = 8, l = 2) and a slightly more complex configuration (n=32,l=2n = 32, l = 2). Herein, nn is the number of neurons within each hidden layer, and ll is the number of hidden layers in NN.

Comparison of Definition of Production and single NN model predictions for two different network configurations, with highlighted boxes indicating selected regions

Figure 12. (a) Single Network: n=8n = 8, l=2l = 2. (b) Single Network: n=32n = 32, l=2l = 2. Comparison of Definition of Production (blue solid line) and single NN model predictions (red dashed line) for two different network configurations. The highlighted boxes indicate regions selected for detailed examination in Figure 13.

Both network configurations effectively capture the overall trend of CsC_s yet display variations in predictive fidelity, particularly in regions characterized by high variability. While the simpler network (Figure 12(a)) demonstrates strong overall agreement with minor deviations, the more complex network (Figure 12(b)) marginally improves alignment, suggesting enhanced capability to represent nuanced variations in the flow data.

To better illustrate specific model behaviors, zoomed-in views of highlighted regions from the main predictions are shown in Figure 13. These provide insights into the network’s local performance and areas of notable discrepancies.

Panel (a)

(a) n=8,l=2n = 8, l = 2, Zoomed Area 1

Panel (b)

(b) n=32,l=2n = 32, l = 2, Zoomed Area 1

Panel (c)

(c) n=8,l=2n = 8, l = 2, Zoomed Area 2

Panel (d)

(d) n=32,l=2n = 32, l = 2, Zoomed Area 2

Panel (e)

(e) n=8,l=2n = 8, l = 2, Zoomed Area 3

Panel (f)

(f) n=32,l=2n = 32, l = 2, Zoomed Area 3

Figure 13. Detailed zoomed-in views of selected regions from Figure 12, highlighting the specific prediction strengths and deviations of the two NN configurations.

These detailed views clarify how different network architectures respond to local fluctuations in turbulent flow data. The simpler network occasionally struggles with highly variable regions, resulting in visible prediction errors (Figure 13a, Figure 13c, Figure 13e). Conversely, the more complex network generally improves local predictions but occasionally shows larger errors, highlighting a trade-off between network complexity and predictive robustness.

Quantitative assessments of the model performance for various configurations are summarized in Table VII, where networks with fewer neurons and moderate depth exhibit favorable MSE values, indicating their potential for better generalization across the complex hurricane boundary layer flows. However, this better generalization feature is not favored in negative Smagorinsky coefficient CsC_s prediction, which is the key to model energy backscatter. Note that in the work by Guan et al.[18] with CNNs directly modeling SGS stresses, it has been demonstrated that an accurate evaluation of SGS stresses that contribute to energy backscatter is of prominent importance in reliable a posteriori model tests. We also mention that the runs presented in Table VII represent instantaneous model evaluations without explicitly setting target MSE values; further improvements in predictive performance can be achieved by specifying desired target MSE thresholds and employing ensemble techniques.

Neurons (nn)Hidden Layers (ll)Test MSE
823.57×10−63.57 \times10^{-6}
834.42×10−64.42 \times10^{-6}
841.82×10−61.82 \times10^{-6}
1621.13×10−51.13 \times10^{-5}
1633.51×10−53.51 \times10^{-5}
1642.82×10−52.82 \times10^{-5}
3222.87×10−42.87 \times10^{-4}
3235.59×10−65.59 \times10^{-6}

Table VII. Test Data MSE for All Single Network Configurations

Ensemble Neural Network Performance

An ensemble NN approach was adopted to enhance accuracy and stability in predicting the Smagorinsky constant (CsC_s). Figure 14 compares the true CsC_s values (solid blue line) with predictions

Comparison of true values and ensemble network predictions for the first representative configuration

(a) (a) Ensemble Network: n=8n = 8, l=2l = 2

Comparison of true values and ensemble network predictions for the second representative configuration

(b) (b) Ensemble Network: n=32n = 32, l=3l = 3

Figure 14. Comparison of true values (blue) and ensemble network predictions (red) for two representative configurations. Highlighted boxes indicate zoomed regions explored in Figure 15.

from ensemble NNs (red dashed line) for two representative configurations: n=8,l=2n = 8, l = 2 and n=32,l=3n = 32, l = 3. The ensemble method significantly reduces prediction variability and aligns closely with the actual data throughout the test dataset. This improved alignment emphasizes the ensemble’s robustness in capturing both general trends and localized fluctuations of CsC_s.

The effectiveness of ensemble networks becomes clearer through detailed inspections presented in Figure 15, where zoomed-in views of selected regions illustrate significant improvements in local predictions. For example, Figure 15(a) and Figure 15(b) highlight enhanced predictive consistency at a challenging local peak around test data indices 1200–1400. Similarly, Figure 15(c) and Figure 15(d), depicting the data indices around 3500–3700, demonstrate the ensemble’s capability to capture complex local variations with reduced deviations compared to single network configurations. Finally, Figure 15(e) and Figure 15(f) illustrate improved accuracy in ensemble predictions within highly fluctuating regions (indices around 4200–4400).

Figure 15. (a) n=8n = 8, l=2l = 2, Zoomed Area 1; (b) n=32n = 32, l=3l = 3, Zoomed Area 1; (c) n=8n = 8, l=2l = 2, Zoomed Area 2; (d) n=32n = 32, l=3l = 3, Zoomed Area 2; (e) n=8n = 8, l=2l = 2, Zoomed Area 3; (f) n=32n = 32, l=3l = 3, Zoomed Area 3. Detailed view of selected regions highlighting improved accuracy and reliability of ensemble NN predictions compared to single NN models.

The quantitative analysis presented in Table VIII further emphasizes the advantages of ensemble NNs. Ensembles consistently achieved significantly lower MSE values across all tested configurations than single networks. The configuration with 32 neurons and three hidden layers achieved the lowest MSE (2.49×10−82.49 \times10^{-8}). The ensemble predictions presented here were conducted using MATLAB, which implements bootstrap aggregating (bagging). That trains multiple NNs on different bootstrap subsets of the dataset, each optimized using Bayesian regularization (‘trainbr’), and records individual network MSE. Subsequently, the script aggregates predictions from networks meeting a specified MSE threshold (1×10−81 \times10^{-8}), taking their median to enhance robustness. The resulting ensemble significantly reduces prediction errors compared to individual networks, notably achieving an MSE as low as 2.49×10−82.49 \times10^{-8}.

Neurons (nn)Hidden Layers (ll)Ensemble Test MSE
822.75×10−62.75 \times10^{-6}
831.61×10−61.61 \times10^{-6}
841.02×10−61.02 \times10^{-6}
1625.87×10−75.87 \times10^{-7}
1636.49×10−76.49 \times10^{-7}
1645.29×10−75.29 \times10^{-7}
3223.73×10−73.73 \times10^{-7}
3232.49×10−82.49 \times10^{-8}

Table VIII. Test Data MSE for All Ensemble Network Configurations (10 Networks)

The results conclusively demonstrate that ensemble NNs significantly outperform single-network configurations, primarily through their ability to average out individual model biases and variances. This ability is especially critical in turbulence modeling, where capturing detailed and nuanced variations in data is vital for accurate predictions. Therefore, ensemble methods represent a powerful tool in operational hurricane boundary-layer simulations, providing enhanced prediction reliability and accuracy.

Comparison of Machine Learning Models and Dynamic Smagorinsky Model

The coefficient CsC_s in the standard Smagorinsky model is considered constant in the computational domain. To overcome these limitations, dynamic models change their parameters in real time and space to reflect the flow condition. Because of this idea, the dynamic Smagorinsky model (DSM) emerged. Germano et al.[43] improved the basic Smagorinsky model by dynamically adjusting the Smagorinsky constant CsC_s based on the flow’s current condition, rather than establishing a fixed value in advance. This dynamic approach involves applying a test filter {⋅~}\{\widetilde{\cdot}\} to the LES data. The LES data is then utilized to compute the resolved stress tensor.

Lij=Tij−τ~ij=ui~uj~−uiuj~(7)\mathcal{L}_{ij}=T_{ij}-\widetilde{\tau}_{ij}=\widetilde{u_i}\widetilde{u_j}-\widetilde{u_i u_j} \tag*{(7)}

Assuming that the same functional form can be used to parametrize both TijT_{ij} and τij\tau_{ij} (the Smagorinsky model, for example), let MijM_{ij} and mijm_{ij} be the models for the anisotropic parts of TijT_{ij} and τij\tau_{ij}:

τij−(δij/3)τkk≃mij=2CΔ‾2∣S‾∗∣S‾ij∗,(8)\tau_{ij}-\left(\delta_{ij}/3\right)\tau_{kk}\simeq m_{ij}=2C\overline{\Delta}^{2}\lvert\overline{S}^{*}\rvert\overline{S}^{*}_{ij}, \tag*{(8)}
Tij−(δij/3)Tkk≃Mij=2CΔ‾~2∣S‾~∗∣S‾~ij∗,(9)T_{ij}-\left(\delta_{ij}/3\right)T_{kk}\simeq M_{ij}=2C\widetilde{\overline{\Delta}}^{2}\lvert\widetilde{\overline{S}}^{*}\rvert\widetilde{\overline{S}}^{*}_{ij}, \tag*{(9)}

where

∣S‾ij∗∣=12(∂ui‾∂xj+∂uj‾∂xi−23∂uk‾∂xkδij),∣S‾~∗∣=2S‾~mn∗S‾~mn∗,\lvert\overline{S}^{*}_{ij}\rvert=\frac{1}{2}\left(\frac{\partial\overline{u_i}}{\partial x_j}+\frac{\partial\overline{u_j}}{\partial x_i}-\frac{2}{3}\frac{\partial\overline{u_k}}{\partial x_k}\delta_{ij}\right),\qquad\lvert\widetilde{\overline{S}}^{*}\rvert=\sqrt{2\widetilde{\overline{S}}^{*}_{mn}\widetilde{\overline{S}}^{*}_{mn}},

Δ‾\overline{\Delta} is the characteristic filter width associated with G‾\overline{G}, and Δ‾~\widetilde{\overline{\Delta}} is the filter width associated with G‾~\widetilde{\overline{G}}.

Now, at first subtracting filtered (8) from (9), and contracting by S‾ij∗\overline{S}^{*}_{ij} gives

[Tij−(δij/3)Tkk−[τ~ij−(δij/3)τ~kk]]S‾ij∗=2C{Δ‾~2∣S‾~∗∣S‾~ij∗S‾ij∗−Δ‾2∣S‾∗∣S‾ij∗S‾ij∗}\left[T_{ij}-\left(\delta_{ij}/3\right)T_{kk}-\left[\widetilde{\tau}_{ij}-\left(\delta_{ij}/3\right)\widetilde{\tau}_{kk}\right]\right]\overline{S}^{*}_{ij}=2C\left\{\widetilde{\overline{\Delta}}^{2}\lvert\widetilde{\overline{S}}^{*}\rvert\widetilde{\overline{S}}^{*}_{ij}\overline{S}^{*}_{ij}-\overline{\Delta}^{2}\lvert\overline{S}^{*}\rvert\overline{S}^{*}_{ij}\overline{S}^{*}_{ij}\right\}
[Tij−τ~ij]S‾ij∗−δij3[Tkk−τ~kk]S‾ij∗=2C{Δ‾~2∣S‾~∗∣S‾~ij∗S‾ij∗−Δ‾2∣S‾∗∣S‾ij∗S‾ij∗}.\left[T_{ij}-\widetilde{\tau}_{ij}\right]\overline{S}^{*}_{ij}-\frac{\delta_{ij}}{3}\left[T_{kk}-\widetilde{\tau}_{kk}\right]\overline{S}^{*}_{ij}=2C\left\{\widetilde{\overline{\Delta}}^{2}\lvert\widetilde{\overline{S}}^{*}\rvert\widetilde{\overline{S}}^{*}_{ij}\overline{S}^{*}_{ij}-\overline{\Delta}^{2}\lvert\overline{S}^{*}\rvert\overline{S}^{*}_{ij}\overline{S}^{*}_{ij}\right\}.

For incompressible flow Sˉii∗=0\bar{S}^{*}_{ii}=0 so, δij3[Tkk−τˉkk]Sˉij∗=Sˉii∗3[Tkk−τˉkk]=0\frac{\delta_{ij}}{3}[T_{kk}-\bar{\tau}_{kk}]\bar{S}^{*}_{ij}=\frac{\bar{S}^{*}_{ii}}{3}[T_{kk}-\bar{\tau}_{kk}]=0. Therefore, CsC_s can be defined as

Cs=12LijSˉij∗Δ2~∣Sˉij∗∣Sˉij∗Sˉij∗−Δ2∣Sˉij∗∣Sˉij∗(10)C_s=\frac{1}{2}\frac{\mathcal{L}_{ij}\bar{S}^{*}_{ij}}{\widetilde{\Delta^2}|\bar{S}^{*}_{ij}|\bar{S}^{*}_{ij}\bar{S}^{*}_{ij}-\Delta^2|\bar{S}^{*}_{ij}|\bar{S}^{*}_{ij}} \tag*{(10)}

On the other hand, the production-based formulation defines the true CsC_s from the TKE production. The CsC_s evaluation details are given in Sect. III B. This production-based CsC_s serves as the ground truth in our comparative analysis.

Ensemble NN predictions compared with the definition of production formulation and dynamic Smagorinsky model

Figure 16. Ensemble NN predictions compared with the definition of production formulation and dynamic Smagorinsky model.

Next, we compare the predictions of CsC_s using two methods, the dynamic Smagorinsky method, and ensemble NN-based predictions, against the definition of production formulation as true values. The comparison is illustrated through figures highlighting the trends, variations, and zoomed regions for better visualization and interpretation.

Figures 16 and 17 present a side-by-side comparison of the CsC_s predicted by three different approaches: the definition of production formulation (considered as the ground truth, shown in solid blue), the DSM (shown in solid green), and the ensemble NN model (shown in dashed red). NNs in an ensemble configuration offer a promising signed CsC_s prediction. By leveraging training on large datasets, they effectively interpolate between the smoothness of the production formulation and the adaptability of the dynamic Smagorinsky method. Their predictions closely follow the dynamic Smagorinsky model while mitigating its extreme fluctuations, making them a more robust alternative for SGS modeling. The ability of NNs to generalize well across different turbulence regimes ensures their viability as a scalable and computationally efficient solution for atmospheric simulations, thus bridging the gap between traditional turbulence models and modern data-driven approaches.

To better understand these differences, Figure 17 zooms into three specific regions of the test data. These close-up views show that while the dynamic model may overshoot or undershoot in certain areas, the NN model stays tightly aligned with the true curve. The ensemble NN effectively filters out noise while preserving important physical features, providing a more stable and accurate estimate of CsC_s across the domain.

Zoomed area 1

(a) Zoomed area 1

Zoomed area 2

(b) Zoomed area 2

Zoomed area 3

(c) Zoomed area 3

Figure 17. Zoomed areas showing NN predictions alongside other methods for CsC_s.

In summary, the ensemble NN is a middle ground between the production formulation’s physical accuracy and the dynamic model’s adaptivity. It offers reliable predictions that avoid the instability of DSM while preserving the variability needed to represent turbulence. These qualities make ensemble NNs a strong candidate for replacing traditional SGS models in large-scale atmospheric simulations.

Conclusions

This study presents a novel framework for SGS turbulence modeling in mesoscale atmospheric simulations by integrating machine learning with physical and geometric invariances. The primary goal of this research is to improve the prediction of the signed Smagorinsky coefficient (CsC_s) by embedding domain-specific knowledge into ML models through tensor invariants and singular value decomposition of local flow field features. These techniques enhance the model’s ability to capture essential turbulence structures while maintaining interpretability and physical consistency. By utilizing high-fidelity LES data, the proposed ML-based approach improves the accuracy of SGS stress modeling, demonstrating significant advantages over traditional turbulence closure methods.

A key contribution of this study is the interpretability of the ML framework. Unlike many existing ML-based SGS models that function as “black-box” predictors, our approach explicitly incorporates physical constraints and geometric invariances. Using tensor invariants ensures that the predictions remain consistent with fundamental turbulence principles, while SVD of local flow features allows for an effective representation of anisotropic turbulence structures. This incorporation of physical knowledge makes the model more robust and transparent, addressing a major challenge in applying ML to scientific modeling.

The comparative analysis of the invariance-embedded ML predictions of signed CsC_s and those with the dynamic Smagorinsky method demonstrate that ensemble NN predictions successfully capture localized turbulence variations similar to the dynamic Smagorinsky model while avoiding the extreme fluctuations that often lead to numerical instabilities.

The study also explores different NN configurations, evaluating their effectiveness in predicting CsC_s. Classification models are shown to effectively distinguish between different turbulence regimes, while regression models accurately predict signed CsC_s values pointwise. The implementation of ensemble learning further enhances prediction stability and accuracy, reducing the variability associated with individual models due to the statistical learning nature of ML. This demonstrates that combining multiple NNs can improve the reliability and generalizability of ML-based turbulence modeling.

This research establishes that ML-based approaches can significantly enhance the accuracy and efficiency of SGS modeling in mesoscale atmospheric flows when designed with physical and geometric considerations. By improving the representation of turbulence in numerical simulations, this work contributes to the ongoing advancement of computational fluid dynamics and atmospheric spheric modeling, with the potential to improve predictive capabilities in hurricane forecasting and other high-impact weather phenomena.

Acknowledgments

The hardware used in the computational studies is part of the UMBC High Performance Computing Facility (HPCF). The facility is supported by the U.S. National Science Foundation through the MRI program (grant nos. CNS-0821258, CNS-1228778, and OAC-1726023) and the SCREMS program (grant no. DMS-0821311), with additional substantial support from the University of Maryland, Baltimore County (UMBC). Hasan and Yu are grateful to Dr. Heng Xiao for the constructive discussions.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request. WRF simulation[44] data used in this study is available on the Research Data Archive of NCAR (https://rda.ucar.edu/datasets/d301000/).

Appendix: Model Performance Metrics for Different Classifier Models

This appendix provides a comparison of performance metrics for various machine learning classifier models, including Logistic Regression (LR), Support Vector Machines (SVMs), Random Forest (RF), Gradient Boosting (GB), and Neural Network (NN), and different configurations tested in this study. The metrics include Accuracy, Precision, Recall, and F1-Score (see the introduction in Section IV A 1), with a particular emphasis on the F1-Score as it balances precision and recall, which is crucial for datasets with potentially imbalanced classes. Different machine learning classifier models are briefly explained in the sections below for completeness. For a detailed description of these models, configurations, and their implementation with data from fine-scale hurricane boundary layer flow simulations, readers are referred to Hasan et al.[54].

Logistic Regression

Logistic Regression[66] is a linear model used for binary classification tasks. It models the probability that a given input xx belongs to a particular class. The logistic function, or sigmoid, is used to map predicted values to probabilities between 0 and 1:

P(y=1∣x)=11+e−(wTx+b)P(y = 1 \mid x) = \frac{1}{1 + e^{-(w^{T}x+b)}}

where ww is the weight vector, xx is the input feature vector, and bb is the bias term. In the context of LR, yy represents the binary outcome or target variable for classification. It can take on one of two possible values, typically labeled as 0 or 1. For our study, in a binary classification problem, y=1y = 1 indicates the large absolute CsC_s values, while y=0y = 0 represents small values. The model aims to predict the probability that y=1y = 1, given the input features xx, using the logistic function to map the predicted value to a range between 0 and 1. The model is trained by minimizing the binary cross-entropy loss, which measures the difference between the predicted probabilities and the true labels:

L(w,b)=−1N∑i=1N(yilog⁡(P(yi∣xi))+(1−yi)log⁡(1−P(yi∣xi)))+12C∥w∥2L(w,b) = -\frac{1}{N}\sum_{i=1}^{N}\left(y_i\log(P(y_i\mid x_i)) + (1-y_i)\log(1-P(y_i\mid x_i))\right) + \frac{1}{2C}\lVert w\rVert^2

where NN is the number of samples, yiy_i represents the true label for sample ii, and ∥⋅∥2\lVert\cdot\rVert^2 denotes the L2L_2 norm operator. The regularization term 12C∥w∥2\frac{1}{2C}\lVert w\rVert^2 penalizes large weights, with the parameter CC controlling the strength of the regularization. A smaller CC value applies stronger regularization, penalizing large weights more heavily and encouraging the model to generalize better by simplifying its decision boundary. This is particularly useful for preventing overfitting in cases with noisy data. Conversely, a larger CC reduces the regularization effect, allowing the model to fit the training data more closely and potentially capture more detailed patterns, albeit at the risk of overfitting.

The weights are optimized using gradient descent to best fit the training data, considering both the data fitting and regularization terms. For LR, the decision line is strongly set by setting the probability threshold, which is usually 0.5, to classify the input effectively:

f(x)={1if P(y=1∣x)≥0.50otherwisef(x) = \begin{cases} 1 & \text{if } P(y = 1 \mid x) \ge0.5 \\ 0 & \text{otherwise} \end{cases}

LR is simple and effective for linearly separable data, making it a popular baseline for classification tasks. By tuning the CC parameter, the model can find a balance between bias and variance, optimizing its performance for specific datasets.

Support Vector Machines

Support Vector Machines[67] were also employed to classify the magnitude of the Smagorinsky coefficient, CsC_s. We tested different kernel functions, specifically the Radial Basis Function (RBF) and linear kernel, varying the regularization parameter (CC) and the kernel coefficient (γ\gamma), to determine their impact on model accuracy. The SVM decision function is given by:

f(x)=sign⁡(∑i=1NαiyiK(xi,x)+b),f(x) = \operatorname{sign}\left(\sum_{i=1}^{N} \alpha_i y_i K(x_i,x) + b\right),

where αi\alpha_i are the Lagrange multipliers, yiy_i are the target labels, K(xi,x)K(x_i,x) is the kernel function, and bb is the bias term. The SVM optimization problem, which includes the regularization parameter CC, is formulated as:

min⁡α12∑i=1N∑j=1NαiαjyiyjK(xi,xj)−∑i=1Nαi,\min_{\alpha} \frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}\alpha_i\alpha_j y_i y_j K(x_i,x_j)-\sum_{i=1}^{N}\alpha_i,

subject to

0≤αi≤C,0 \leq\alpha_i \leq C,
∑i=1Nαiyi=0.\sum_{i=1}^{N}\alpha_i y_i = 0.

The optimal parameters were identified through cross-validation, balancing precision, and recall to achieve robust classification performance. The RBF kernel is particularly useful for handling nonlinear relationships within the data. The RBF kernel function is defined as:

K(xi,x)=exp⁡(−γ∥xi−x∥2),K(x_i,x) = \exp\left(-\gamma\lVert x_i-x\rVert^2\right),

where γ\gamma is a parameter that determines the spread of the kernel, and ∥x−x′∥2\lVert x-x'\rVert^2 is the squared Euclidean distance between the two feature vectors. An equivalent definition involves a parameter γ=12σ2\gamma= \frac{1}{2\sigma^2}, where σ\sigma is a free parameter that controls the width of the Gaussian function. A smaller σ\sigma value results in a narrower Gaussian, focusing the kernel on close points. A bigger σ\sigma value results in a wider Gaussian, which helps the kernel catch connections between distant points. This kernel transforms the data into a higher-dimensional space where it becomes easier to separate classes that are not linearly separable in the original feature space. The RBF kernel can find the complex patterns in the dataset, which helps the model to classify data points more accurately.

Random Forest

Random Forest[68] is an ensemble learning method for classification and regression tasks. It creates multiple decision trees during training and outputs the mode of the classes for classification or the average prediction for regression. This approach reduces the risk of overfitting compared to individual decision trees and increases model robustness.

The prediction from an RF is given by aggregating the predictions from all the trees in the forest, ensuring a reliable and robust outcome:

f(x)=1M∑m=1MTm(x),f(x) = \frac{1}{M} \sum_{m=1}^{M} T_m(x),

where MM is the number of trees, and Tm(x)T_m(x) represents the prediction from the mm-th tree. Each tree is constructed using a bootstrap sample from the original dataset, and random subsets of features are selected for splitting at each node, thus introducing diversity and reducing the correlation between the trees.

The optimization problem for RF involves minimizing the impurity at each node split. The Gini impurity or entropy is often used as the criterion:

G=1−∑k=1Kpk2,G = 1 - \sum_{k=1}^{K} p_k^2,

where pkp_k is the proportion of samples belonging to class kk at a particular node, and KK is the number of classes. For classification tasks, KK is determined by the number of unique class labels in the target variable. For example, if the target variable represents three categories (say, ‘A,’ ‘B,’ and ‘C’), then K=3K = 3. The impurity measure evaluates the “purity” of a node, where a lower value indicates that the node predominantly contains samples from a single class. When splitting the nodes, RF chooses the feature and threshold that result in the greatest reduction in impurity.

The hyperparameters tuned in this study included the number of trees (MM) and the maximum depth of each tree. These hyperparameters were chosen using cross-validation, a technique used to assess how well a model will generalize to an independent data set. This process helps optimize the performance of the classification model.

Gradient Boosting

Gradient Boosting[69] is an ensemble learning technique that builds a model sequentially, combining the strengths of weak learners (typically decision trees) to improve overall performance. Each new model is designed to correct the errors made by the previous models, ensuring a reliable and robust predictive model.

The GB algorithm minimizes a loss function by adding new models that approximate the negative gradient of the loss function concerning the current model predictions. The prediction at iteration mm is given by:

Fm(x)=Fm−1(x)+η⋅hm(x)F_m(x) = F_{m-1}(x) + \eta\cdot h_m(x)

where Fm−1(x)F_{m-1}(x) is the prediction from the previous iteration, hm(x)h_m(x) is the new weak learner (typically a decision tree), and η\eta is the learning rate that controls the contribution of each tree to the final model.

The loss function minimized in GB is typically the mean squared error for regression tasks or the binary cross-entropy for classification tasks:

Lm=∑i=1Nl(yi,Fm(xi))L_m = \sum_{i=1}^{N} l(y_i,F_m(x_i))

where l(yi,Fm(xi))l(y_i,F_m(x_i)) represents the loss for each data point ii.

The hyperparameters tuned in this study included the number of boosting stages, the learning rate (η\eta), and the maximum depth of each tree. These parameters were optimized through cross-validation to achieve the best performance in classifying the signed Smagorinsky coefficient, CsC_s.

Neural Networks

Neural networks[70, 71, 72] are computational models inspired by biological NNs. They consist of interconnected units or nodes, called neurons, organized in layers that transform input data xx into output f(x)f(x) through a series of weighted connections and activation functions.

The output of a single-layer neuron can be represented as:

f(x)=σ(wTx+b)f(x) = \sigma(w^T x+b)

where ww is the weight vector, xx is the input feature vector, bb is the bias term, and σ\sigma is the activation function, such as ReLU or sigmoid. Deep neural networks extend this structure with multiple hidden layers, enabling the model to learn complex, hierarchical data representations.

Training an NN involves optimizing the weights and biases to minimize a loss function, such as binary cross-entropy for classification. The backpropagation algorithm[60] calculates the gradient of the loss concerning each parameter, and gradient descent is used to update the parameters to reduce the loss.

We designed and implemented a feedforward NN using the PyTorch[58] library, exploring a range of fully connected architectures with diverse configurations to evaluate their impact on the model’s performance. We meticulously adjusted key hyperparameters to optimize the model, including the number of hidden layers, nodes per layer, and activation functions. During training, our main focus is minimizing binary cross-entropy loss using backpropagation and gradient descent algorithms. The Binary Cross-Entropy Loss[59] is a commonly used loss function for binary classification tasks. The formula for the Binary Cross-Entropy (BCE) Loss is:

BCE Loss=−1N∑i=1N[yilog⁡(y^i)+(1−yi)log⁡(1−y^i)]\mathrm{BCE\ Loss} = -\frac{1}{N}\sum_{i=1}^{N}\left[y_i\log(\hat{y}_i) + (1-y_i)\log(1-\hat{y}_i)\right]

where NN is the number of samples, yiy_i is the true label for the ii-th sample (0 or 1), and y^i\hat{y}_i is the predicted probability for the ii-th sample, which is the output of the sigmoid function applied to the model’s output. The model’s output is passed through a sigmoid function to produce probabilities, which are then compared to the true labels using the BCE Loss function.

Each figure below shows the performance metrics for a specific classifier model type and configuration, helping to identify trends in how different parameters influence the effectiveness of each classifier, including that for LR in Figure 18, SVM in Figure 19, RF in Figure 20, GB in Figure 21, and NN in Figure 22.

Performance metrics for Logistic Regression classifier configurations

Figure 18. Performance metrics for Logistic Regression classifier configurations. Different values of the regularization parameter (CC) are tested to assess the impact on the model’s ability to classify accurately and generalize effectively.

Performance metrics for Support Vector Machine classifier configurations

Figure 19. Performance metrics for Support Vector Machine classifier configurations. The regularization parameter (CC) and kernel coefficient (γ\gamma) are tuned to explore their influence on the model’s decision boundary and classification accuracy.

Performance metrics for Random Forest classifier configurations

Figure 20. Performance metrics for Random Forest classifier configurations. Various combinations of estimators (est) and maximum depth (depth) are tested. The F1-Score and Recall values demonstrate the model’s sensitivity to positive cases across different configurations.

Performance metrics for Gradient Boosting classifier configurations

Figure 21. Performance metrics for Gradient Boosting classifier configurations. The configurations vary by learning rate (lr) and number of estimators (est). The F1-Score, indicated by the thicker line, highlights the model’s overall balance in handling false positives and negatives across configurations.

Performance metrics for NN classifier configurations

Figure 22. Performance metrics for NN classifier configurations. The model is tested with different activation functions and varying neuron counts (From Table II) to examine their effect on model accuracy, precision, recall, and F1-Score.

References

References

  1. [1]1I. B. Celik, Z. N. Cehreli, and I. Yavuz, “Index of Resolution Quality for Large Eddy Simulations,” Journal of Fluids Engineering 127, 949–958 (2005).
  2. [2]2G. Matheou, “Numerical discretization and subgrid-scale model effects on large-eddy simulations of a stable boundary layer,” Quarterly Journal of the Royal Meteorological Society 142, 3050–3062 (2016).DOI
  3. [3]3E. Bou-Zeid, C. Meneveau, and M. Parlange, “A scale-dependent Lagrangian dynamic model for large eddy simulation of complex turbulent flows,” Physics of Fluids 17, 025105 (2005).
  4. [4]4F. Giraldo and M. Restelli, “A study of spectral element and discontinuous galerkin methods for the navier–stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases,” Journal of Computational Physics 227, 3849–3877 (2008).DOI
  5. [5]5M. B. Hasan, S. R. Guimond, M. L. Yu, S. Reddy, and F. X. Giraldo, “The effects of numerical dissipation on hurricane rapid intensification with observational heating,” Journal of Advances in Modeling Earth Systems 14 (2022).
  6. [6]6M. L. Yu, F. X. Giraldo, M. Peng, and Z. Wang, “Localized artificial viscosity stabilization of discontinuous galerkin methods for nonhydrostatic mesoscale atmospheric modeling,” Monthly Weather Review 143, 4823–4845 (2015).DOI
  7. [7]B. T. Nadiga and D. Livescu, “Instability of the perfect subgrid model in implicit-filtering large eddy simulation of geostrophic turbulence,” Phys. Rev. E 75, 046303 (2007).pubmed.ncbi.nlm.nih.gov/17500989
  8. [8]J. G. Ballouz and N. T. Ouellette, “Tensor geometry in the turbulent cascade,” Journal of Fluid Mechanics 835, 1048–1064 (2018).DOI
  9. [9]M. Linkmann, M. Buzzicotti, and L. Biferale, “Multiscale properties of large eddy simulations: correlations between resolved-scale velocity-field increments and subgrid-scale quantities,” Journal of Turbulence 19, 1–35 (2018).DOI
  10. [10]A. Vela-Martín, “Subgrid-scale models of isotropic turbulence need not produce energy backscatter,” Journal of Fluid Mechanics 937, A14 (2022).
  11. [11]M. F. Jansen and I. M. Held, “Parameterizing subgrid-scale eddy effects using energetically consistent backscatter,” Ocean Modelling 80, 36–48 (2014).
  12. [12]I. Grooms, “Backscatter in energetically-constrained Leith parameterizations,” Ocean Modelling 186, 102265 (2023).DOI
  13. [13]C.-Y. Chang, A. Adcroft, L. Zanna, R. Hallberg, and S. M. Griffies, “Remote versus local impacts of energy backscatter on the north atlantic sst biases in a global ocean model,” Geophysical Research Letters 50, e2023GL105757 (2023).DOI
  14. [14]S. L. Brunton, B. R. Noack, and P. Koumoutsakos, “Machine learning for fluid mechanics,” Annual Review of Fluid Mechanics 52, 477–508 (2020).
  15. [15]K. Duraisamy, “Perspectives on machine learning-augmented reynolds-averaged and large eddy simulation models of turbulence,” Phys. Rev. Fluids 6, 050504 (2021).DOI
  16. [16]P. Gentine, V. Ey ring, and T. Beucler, “Deep learning for the parametrization of subgrid processes in climate models,” in Deep Learning for the Earth Sciences (John Wiley & Sons, Ltd, 2021) Chap. 21, pp. 307–314.DOI
  17. [17]L. Zanna and T. Bolton, “Deep learning of unresolved turbulent ocean processes in climate models,” in Deep Learning for the Earth Sciences (John Wiley & Sons, Ltd, 2021) Chap. 20, pp. 298–306.DOI
  18. [18]Y. Guan, A. Chattopadhyay, A. Subel, and P. Hassanzadeh, “Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning,” Journal of Computational Physics 458, 111090 (2022).DOI
  19. [19]T. Bolton and L. Zanna, “Applications of deep learning to ocean data inference and subgrid parameterization,” Journal of Advances in Modeling Earth Systems 11, 376–399 (2019).DOI
  20. [20]H. Frezat, J. Le Sommer, R. Fablet, G. Balarac, and R. Lguensat, “A posteriori learning for quasi-geostrophic turbulence parametrization,” Journal of Advances in Modeling Earth Systems 14, e2022MS003124 (2022).
  21. [21]Y. Cheng, M. G. Giometto, P. Kauffmann, L. Lin, C. Cao, C. Zupnick, H. Li, Q. Li, Y. Huang, R. Abern athey, and P. Gentine, “Deep learning for subgrid-scale turbulence modeling in large-eddy simulations of the convective atmospheric boundary layer,” Journal of Advances in Modeling Earth Systems 14, e2021MS002847 (2022).
  22. [22]A. P. Guillaumin and L. Zanna, “Stochastic-deep learning parameterization of ocean momentum forcing,” Journal of Advances in Modeling Earth Systems 13, e2021MS002534 (2021).DOI
  23. [23]J. Yuval and P. A. O’Gorman, “Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions,” Nature Communications 11 (2020), 10.1038/s41467-020-17142-3.DOI
  24. [24]Y. Guan, A. Subel, A. Chattopadhyay, and P. Hassanzadeh, “Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate les,” Physica D: Nonlinear Phenomena 443, 133568 (2023).DOI
  25. [25]M. B. Hasan, M. Yu, and H. Xiao, “Sub-grid scale modeling of meso-scale hurricane boundary layer flows using machine learning,” in AIAA SCITECH 2023 Forum (2023) p. 2487.DOI
  26. [26]X.-H. Zhou, J. Han, and H. Xiao, “Frame-independent vector-cloud neural network for non-local constitutive modelling on arbitrary grids,” Computer Methods in Applied Mechanics and Engineering (2022).
  27. [27]J. Ling, R. Jones, and J. Templeton, “Machine learning strategies for systems with invariance properties,” Journal of Computational Physics 318, 22–35 (2016).DOI
  28. [28]C. Meneveau and J. Katz, “Scale-invariance and turbulence models for large-eddy simulation,” Annual Review of Fluid Mechanics 32, 1–32 (2000), https://doi.org/10.1146/annurev.fluid.32.1.1.
  29. [29]P. Sagaut, Large Eddy Simulation for Incompressible Flows: An Introduction, 3rd ed. (Springer, 2006).
  30. [30]G. B. A. Vollant and C. Corre, “Subgrid-scale scalar flux modelling based on optimal estimation theory and machine-learning procedures,” Journal of Turbulence 18, 854–878 (2017).DOI
  31. [31]A. Vreman, B. Geurts, and H. Kuerten, “Subgrid-modeling in les of compressible flow,” Applied Scientific Research 54, 191–203 (1995).
  32. [32]D. Carati, S. Ghosal, and P. Moin, “On the representation of backscatter in dynamic localization models,” Physics of Fluids 7, 606–616 (1995).DOI
  33. [33]R. M. Kerr, J. A. Domaradzki, and G. Barbier, “Small-scale properties of nonlinear interactions and subgrid-scale energy transfer in isotropic turbulence,” Physics of Fluids 8, 197–208 (1996).DOI
  34. [34]P. J. Mason and D. J. Thomson, “Stochastic backscatter in large-eddy simulations of boundary layers,” Journal of Fluid Mechanics 242, 51–78 (1992).DOI
  35. [35]J. A. Domaradzki, “Large eddy simulations of high reynolds number turbulence based on inter-scale energy transfer among resolved scales,” Phys. Rev. Fluids 6, 044609 (2021).DOI
  36. [36]M. Nabavi and J. Kim, “Optimising subgrid-scale closures for spectral energy transfer in turbulent flows,” Journal of Fluid Mechanics 982, A18 (2024).DOI
  37. [37]A. Beck, D. Flad, and C.-D. Munz, “Deep neural networks for data-driven LES closure models,” Journal of Computational Physics 398, 108910 (2019).DOI
  38. [38]Z. Zhou, S. Wang, X. Yang, and G. Jin, “A structural subgrid-scale model for the collision-related statistics of inertial particles in large-eddy simulations of isotropic turbulent flows,” Physics of Fluids 32, 095103 (2020).DOI
  39. [39]C. Xie, Z. Yuan, and J. Wang, “Artificial neural network-based nonlinear algebraic models for large eddy simulation of turbulence,” Physics of Fluids 32, 115101 (2020).DOI
  40. [40]C. Xie, J. Wang, and W. E, “Modeling subgrid-scale forces by spatial artificial neural networks in large eddy simulation of turbulence,” Phys. Rev. Fluids 5, 054606 (2020).DOI
  41. [41]S. Sroka and S. Guimond, “Organized kinetic energy backscatter in the hurricane boundary layer from radar measurements,” Journal of Fluid Mechanics 924 (2021).
  42. [42]U. Piomelli and E. Balaras, “Wall-layer models for large-eddy simulations,” Annual Review of Fluid Mechanics 34, 349–374 (2002).
  43. [43]M. Germano, U. Piomelli, P. Moin, and W. H. Cabot, “A dynamic subgrid-scale eddy viscosity model,” Physics of Fluids A: Fluid Dynamics 3, 1760–1764 (1991).
  44. [44]H. Ren, J. Dudhia, and H. Li, “Large-eddy simulation of idealized hurricanes at different sea surface temperatures,” Journal of Advances in Modeling Earth Systems 12, e2020MS002057 (2020).DOI
  45. [45]W. C. Skamarock, J. B. Klemp, J. Dudhia, O. Gill, Z. Liu, J. Berner, W. Wang, G. Powers, G. Duda, D. M. Barker, and X. yu Huang, “A description of the advanced research wrf model version 4,” NCAR Technical Notes (2021), http://dx.doi.org/10.5065/1dfh-6p97.
  46. [46]R. Rotunno, Y. Chen, W. Wang, C. Davis, J. Dudhia, and G. J. Holland, “Large-eddy simulation of an idealized tropical cyclone,” Bulletin of the American Meteorological Society 90, 1783 – 1788 (2009).DOI
  47. [47]47 A. N. Kolmogorov, “The local structure of turbulence in incompressible viscous fluid for very large reynolds numbers,” Proceedings: Mathematical and Physical Sciences 434, 9–13 (1991).
  48. [48]48 S. Pope, “Turbulent flows,” (2000), https://doi.org/10.1017/CBO9780511840531.
  49. [50]50 D. C. Wilcox, Turbulence Modeling for CFD (2nd ed.) (DCW Industries, La Cañada, 2008).
  50. [52]52 J. Wang, J.-L. Wu, and H. Xiao, “Physics-informed machine learning approach for reconstructing reynolds stress modeling discrepancies based on dns data,” Physical Review Fluids 2, 034603 (2017).
  51. [53]53 J.-L. Wu, H. Xiao, and E. Paterson, “Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework,” Phys. Rev. Fluids 3, 074602 (2018).
  52. [54]54 M. B. Hasan, M. Yu, and T. Oates, “Comparison of several neural network-enhanced sub-grid scale stress models for meso-scale hurricane boundary layer flow simulation,” in AIAA SCITECH 2025 Forum (2025).DOI
  53. [55]55 B. D. Tracey, K. Duraisamy, and J. J. Alonso, “Application of supervised learning to quantify uncertainties in turbulence and combustion modeling,” 51st AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition (2013).DOI
  54. [56]56 H. E. Willoughby, “Temporal changes of the primary circulation in tropical cyclones,” Journal of the Atmospheric Sciences 47, 242–264 (1990).doi.org/10.1175/1520-0469(1990)047<0242:tcotpc>2.0.co;2
  55. [57]57 F. D. Marks, P. G. Black, M. T. Montgomery, and R. W. Burpee, “Structure of the eye and eyewall of hurricane hugo (1989),” Monthly Weather Review 136, 1237–1259 (2008).DOI
  56. [58]58 A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems, Vol. 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Curran Associates, Inc., 2019).arxiv.org/abs/1912.01703
  57. [59]59 I. J. Good, “Rational decisions,” Journal of the Royal Statistical Society. Series B (Methodological) 14, 107–114 (1952).
  58. [60]D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 323, 533–536 (1986).
  59. [61]D. M. Hawkins, “The problem of overfitting,” Journal of Chemical Information and Computer Sciences 44, 1–12 (2004).
  60. [62]N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research 15, 1929–1958 (2014).
  61. [63]F. D. Foresee and M. T. Hagan, “Gauss-newton approximation to bayesian learning,” Proceedings of International Conference on Neural Networks (ICNN’97) 3, 1930–1935 vol.3 (1997).
  62. [64]D. J. C. MacKay, “A Practical Bayesian Framework for Backpropagation Networks,” Neural Computation 4, 448–472 (1992), https://direct.mit.edu/neco/article-pdf/4/3/448/812348/neco.1992.4.3.448.pdf.DOI
  63. [65]M. B. Hasan, M. Yu, and T. Oates, “Evaluating machine learning-enhanced sub-grid scale stress models with invariance embedding for meso-scale hurricane boundary layer flows,” in ASME Fluids Engineering Division Summer Meeting (FEDSM) (2025 (Accepted)).DOI
  64. [66]D. W. Hosmer, S. Lemeshow, and R. X. Sturdivant, Applied Logistic Regression (Wiley, 2013).
  65. [67]C. Cortes and V. Vapnik, “Support-vector networks,” Machine learning 20, 273–297 (1995).DOI
  66. [68]L. Breiman, “Random forests,” Machine Learning 45, 5–32 (2001).
  67. [69]J. H. Friedman, “Greedy function approximation: A gradient boosting machine.” The Annals of Statistics 29, 1189 – 1232 (2001).DOI
  68. [70]Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature 521, 436–444 (2015).DOI
  69. [71]I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016).
  70. [72]J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks 61, 85–117 (2015).

Paper details

vlnc:W_fnpfbynf
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents