betaFeedback

WorksPhysics and Astronomy

Neural network scaling and generalization for 3D strain fields in heterogeneous materials

Abstract

Computational homogenization and coupled multiscale simulations of heterogeneous materials require repeated solution of mechanical fields on Representative Volume Elements, which becomes prohibitively expensive for high-resolution three-dimensional microstructures. Machine-learning surrogates can accelerate these computations, yet most existing approaches predict fields for fixed macroscopic loadings and provide limited guidance on how to scale network architectures for high-resolution 3D tensor-field prediction. This work presents a U-Net surrogate trained to predict the full 36-component strain localization tensor field in voxelized concrete mesostructures at resolutions up to 2563256^3. Training localization tensors are generated with an FFT-based linear elastic solver, and an extensive hyperparameter optimization is performed to derive scaling rules and an accuracy-speed trade-off analysis. The results show that only network width and depth need to be scaled with problem complexity, while several conventional U-Net design choices, such as doubling channels after downsampling, are suboptimal for this problem. The final neural network achieves a Mean Absolute Error of 1.118% on both the validation and the held-out test sets, and an inference speedup of approximately 8.7x over the FFT-based solver on an Nvidia H100 GPU. Generalization experiments reveal transfer across aggregate volume fractions and highlight the importance of training on diverse morphologies. To safeguard reliability, a Maximum Mean Discrepancy metric computed on internal feature maps is proposed for detecting out-of-distribution microstructures without costly reference computations.

Preprint

This version has not been peer-reviewed.

Citation:

L. Fourel, P. Devianne, M. Debouvry, T. Destruhaut, R. Gunn, A. Gupta, L. Graham-Brady, J.-F. Molinari. Neural network scaling and generalization for 3D strain fields in heterogeneous materials. 2026.

©2026. This manuscript version is made available under the CC-BY 4.0 license.

https://creativecommons.org/licenses/by/4.0/

Introduction

In engineering applications, solid materials are generally considered homogeneous at the macroscopic scale, although most of them are heterogeneous at a finer scale. For instance, concrete is composed of aggregates embedded in a cementitious matrix and these two phases possess different mechanical properties, leading to non-uniform stress and strain fields that govern macroscopic performance and durability. The same principle applies to fiber-reinforced composites, and polycrystalline materials.

Computational methods such as the Finite Element Method (FEM) and Fast Fourier Transform (FFT) based solvers [1, 2] can be used to compute stress and strain fields in heterogeneous materials. These fields are generally used for homogenization or coupled multiscale simulation purposes. In one-way homogenization, the objective is to derive a macroscopic effective material behavior that will be assigned to the equivalent homogeneous material in macroscopic models. In coupled multiscale approaches, such as FE2^2 [3] and FE-FFT [4], there is no macroscopic constitutive law, and the material behavior is evaluated at each macroscopic integration point by computing and averaging the stress in a heterogeneous Representative Volume Element (RVE) subjected to the macroscopic strain. The multiscale methods can require a very large number of RVE computations depending on the size of the macroscopic mesh and the number of iterations necessary to reach convergence between the two scales (in staggered solving schemes). Unfortunately, for microstructures (or mesostructures) with complex geometries, the computational cost can be very large due to the high resolution required to resolve the fine details. Due to the numerous RVE computations and the potentially high computational cost of each of them, multiscale simulation approaches often lead to infeasible computational times even though these approaches have a tremendous potential to improve the accuracy of simulations by bridging multiple scales, which helps understand and predict complex physical phenomena involving the interaction between a macroscopic problem and microscopic (or mesoscopic) features.

In recent years, Machine Learning (ML) has emerged as a promising paradigm for accelerating the computations of stress and strain fields in heterogeneous materials [5, 6]. Instead of computing these fields on-the-fly for each new microstructure or new loading condition, a dataset is precomputed using traditional numerical methods (generally FEM or FFT-based solvers), an ML model is trained on this dataset, and then used as a surrogate for the numerical model. The computation using the surrogate is often faster than the original at the expense of an additional approximation error. This approach is particularly beneficial for multiscale simulations requiring numerous RVE computations for which the computational cost is critical, although ML surrogate models can be used on their own, to evaluate stress or strain fields in new microstructures and eventually derive macroscopic effective behaviors. One important limitation is that the generalization outside the training data distribution is not guaranteed.

In 2015, Liu et al. [7] trained multiple ML models including decision trees, random forests and support vector regression models to predict linear elastic strain fields in 3D microstructures and found that random forest was the best approach for their configuration, with a Mean Absolute Error (MAE) of 13 %. In 2019, Yang et al. [8] discovered that neural networks, and particularly Convolutional Neural Networks (CNNs) were even better than random forest, reaching 3 % MAE on the same task. Khanolkar et al. [9] showed CNN capabilities on a wide range of microstructures. A CNN variation, the U-Net architecture [10], first developed for medical image segmentation purposes, was found to be very efficient at predicting stress and strain fields in composites [11, 12, 13, 14] and polycrystalline microstructures [15, 16, 17]. Yang et al. [18] used a generative adversarial training approach on a U-Net for similar tasks, and Monteiro Fernandes et al. [19] compared training based on a physics-informed loss function (minimization of the stress divergence) and training based on simulation data. Other studies have trained U-Nets on more complex microstructures with varying morphologies [20, 21] and volume fractions [22]. In addition to synthetic microstructures, CNNs are also able to predict stress fields in microstructures measured by X-ray tomography [23]. Researchers have successfully trained U-Nets to predict not only strain fields but also fracture patterns [24, 25]. Alternative architectures, including graph neural networks [26, 27], Fourier neural operators [28], difference-based networks [29], and hybrid convolution-attention U-Nets [30, 31] have also shown good capabilities.

Most existing ML surrogates for heterogeneous materials predict microscopic stress or strain fields for one or a few fixed macroscopic loadings, which limits their usefulness in multiscale frameworks where arbitrary loading paths must be evaluated. Predicting the full strain localization tensor instead provides a more general representation: a single network forward pass yields a microstructure-specific linear operator from which strain and stress fields can be reconstructed for any macroscopic strain in the linear elastic regime. The same concept extends to tangent operators in nonlinear problems, provided the prediction is conditioned on the current strain state. This tensorial formulation is therefore both physically grounded and computationally advantageous. Furthermore, while U-Nets have repeatedly been shown to perform well on field-prediction tasks, the literature lacks a systematic investigation of how their architecture should be scaled for high-resolution three-dimensional tensor fields, and how hyperparameters should be chosen to balance predictive accuracy against the computational speed required in multiscale simulations. The present work addresses these gaps by (1) training a parametric 3D U-Net to predict the complete 36-component strain localization tensor in 64364^3 to 2563256^3 concrete mesostructures, (2) performing an extensive hyperparameter optimization to derive robust scaling rules and an accuracy-speed trade-off analysis, and (3) assessing the generalization limits of such surrogates when the training microstructural distribution is varied. The resulting model is intended as a fast, data-driven substitute for RVE solvers in coupled multiscale simulations, while highlighting the practical trade-offs that must be navigated when deploying machine-learning surrogates in computational mechanics.

Methods

Task

The neural network is trained to predict the strain localization tensor field AA based on the microstructure. This tensor governs the mapping from the macroscopic strain εˉ\bar{\varepsilon} to the microscopic strain ε\varepsilon:

ε(x)=A(x):εˉ(1)\varepsilon(x) = A(x) : \bar{\varepsilon} \tag*{(1)}

Consequently, the microscopic stress σ\sigma is obtained via:

σ(x)=C(x):A(x):εˉ(2)\sigma(x) = C(x) : A(x) : \bar{\varepsilon} \tag*{(2)}

Determining the localization tensor enables homogenization and provides access to local strain and stress fields. While AA is constant for linear elastic materials, the same construction yields a tangent operator in nonlinear regimes, where it depends on the strain state or material history and maps increments of macroscopic strain to increments of microscopic strain. The nonlinear case is, however, substantially more demanding: the operator must be conditioned on the current state, and it must be identified from perturbed rather than unit loadings. This study focuses exclusively on the linear elastic regime.

The localization tensor AA exhibits the following properties:

  • AA is a fourth-order tensor field dependent on the spatial position xx.

  • Due to the symmetry of εˉ\bar{\varepsilon} and ε\varepsilon, AA possesses minor symmetries. It can thus be represented as a 6×66 \times6 matrix field using Voigt-Mandel notation.

  • Unlike the local stiffness tensor CC, AA generally lacks major symmetry; therefore, it retains 36 independent components rather than reducing to 21.

  • The volume average of A(x)A(x) equals the identity tensor II.

  • The volume average of C(x):A(x)C(x) : A(x) yields the effective stiffness tensor Cˉ\bar{C}, relating macroscopic stress to strain: σˉ=Cˉ:εˉ\bar{\sigma} = \bar{C} : \bar{\varepsilon}.

Given the 36 components of the localization tensor, the microscopic strain and stress fields (both 6-component fields) can be derived for arbitrary macroscopic loading using Equation (1) or Equation (2). The neural network input is a scalar phase map of the microstructure, and the output is the corresponding 36-component tensor field.

To train and evaluate the surrogate, we first generate a reference dataset of microstructures and corresponding localization tensors.

Data

This study employs synthetic data generated via computational algorithms. However, the proposed methodology is directly applicable to experimental data obtained from real specimens.

Microstructure generation

We investigate concrete mesostructures, though the term microstructure is also used hereafter for generality. These mesostructures comprise two phases: cement paste (matrix) and aggregates (particles).

Periodic voxel-based mesostructures were generated to ensure compatibility with the FFT-based solver used to compute the strain localization tensor (see subsubsection 2.2.2). Generation was performed using the open-source package PyCMG [32], which implements a random sequential adsorption algorithm. We varied the resolution, aggregate volume fraction (VF), and aggregate morphology, as illustrated in Figure 1 and Figure 2. All mesostructures share an aggregate size distribution following a power law with exponent 0.5 (Fuller curve), with equivalent diameters ranging from 5 % to 25 % of the domain size. The problem is formulated in dimensionless units by normalizing all characteristic lengths with respect to the domain size.

Concrete mesostructures at three different resolutions

Figure 1. Concrete mesostructures generated at three different resolutions. Samples correspond to an aggregate volume fraction (VF) of 20 % and a shape ratio of 100 %.

Concrete mesostructures with varying aggregate volume fractions and shape ratios

Figure 2. Mesostructures generated with varying aggregate volume fractions (VF) and shape ratios. Samples correspond to resolution 2563256^{3}.

Resolutions were selected as powers of 2 to facilitate downsampling: 64364^{3}, 1283128^{3}, and 2563256^{3}. Aggregate volume fractions were set to 20 % and 40 %. These values are lower than typical concrete fractions to account for fine aggregates (smaller than 5 % of the domain size) that cannot be resolved at the lowest resolution and are effectively homogenized into the cement matrix. The aggregate shape parameter defines the aspect ratio (ratio of largest to smallest axes). A ratio of 150 % yields elongated aggregates, whereas 100 % corresponds to equiaxed polyhedra (see Figure 2). For each resolution and parameter combination, 256 random samples were generated.

With the mesostructures in place, the next step is to compute the associated strain localization tensors.

Strain computation

To generate the output examples for the neural network training, the strain localization tensor field A(x)A(x) was computed for each generated microstructure using an FFT-based solver [1, 2]. This approach lever ages the periodicity of the generated voxel grids to solve the Lippmann–Schwinger integral equation associated with the linear elastic equilibrium problem efficiently.

The local stiffness tensor C(x)\mathcal{C}(x) is defined piecewise based on the phase map. The cement matrix is assigned an isotropic stiffness tensor CmC_m, while the aggregates are assigned a stiffer isotropic tensor CaC_a. In dimensionless units, the Young’s modulus of the matrix is set to Em=1E_m = 1, and the aggregate modulus is set to Ea=5E_a = 5, representing a typical contrast found in concrete mesostructures. The Poisson’s ratio is fixed at ν=0.2\nu= 0.2 for both phases.

To fully reconstruct the 6×66 \times6 localization matrix at every voxel, the unit cell is subjected to six linearly independent macroscopic strain loadings εˉ(k)\bar{\varepsilon}^{(k)} (k=1,…,6k = 1, \ldots, 6), corresponding to the basis vectors in Voigt-Mandel notation:

εˉ(1)=[100000],…,εˉ(6)=[000001].(3)\bar{\varepsilon}^{(1)} = \begin{bmatrix} 1\\ 0\\ 0\\ 0\\ 0\\ 0 \end{bmatrix} ,\ldots,\bar{\varepsilon}^{(6)} = \begin{bmatrix} 0\\ 0\\ 0\\ 0\\ 0\\ 1 \end{bmatrix}. \tag*{(3)}

For each loading case kk, the solver computes the resulting microscopic strain field ε(k)(x)\varepsilon^{(k)}(x). The kk-th column of the localization matrix A(x)A(x) in Voigt-Mandel notation is then directly identified as the vector ε(k)(x)\varepsilon^{(k)}(x). By concatenating the results from all six loadings, we obtain the complete tensor field A(x)∈R36A(x) \in\mathbb{R}^{36} for each sample.

The FFT-based solver used in this work relies on the original Lippmann–Schwinger strain formulation. Spatial differential operators are discretized using a forward-backward finite difference scheme, reducing the typical FFT-based solver artifacts. The conjugate gradient method is employed with a convergence criterion based on the equilibrium residual. The iterative process is terminated when the L2L_2-norm of the residual falls below 10−610^{-6}. All simulations were performed on a high-performance computing cluster, using Nvidia H100 SMX5 94 GB GPUs.

A representative strain localization tensor field is visualized in Figure 3. The 36 independent components are organized into a 6×66 \times6 matrix using Voigt-Mandel notation. Consistent with the formulation in subsection 2.1, each column corresponds to the microscopic strain response induced by a specific unit macroscopic loading, while the rows represent the individual strain components.

Visualization of the strain localization tensor field

Figure 3. Visualization of the strain localization tensor field for a representative mesostructure (64364^{3} voxels, 20 % aggregate volume fraction, shape ratio 1.0). Each column gives the microscopic strain induced by a unit macroscopic loading, with rows denoting individual strain components. The tensor is dimensionless, mapping the macroscopic strain εˉ\bar{\varepsilon} to the microscopic strain ε(x)\varepsilon(x).

Data splitting

The generated dataset comprising 256 samples for each configuration (resolution, volume fraction, and shape ratio) was partitioned into training, validation, and test sets to ensure robust model evaluation and prevent data leakage. A stratified splitting strategy was employed to maintain a consistent distribution of microstructural parameters across all subsets. Specifically, 75 % of the data (192 samples per configuration) was allocated for training the neural network weights. The validation set, comprising 12.5 % (32 samples), was used for monitoring convergence and hyperparameter tuning. The remaining 12.5 % (32 samples) constituted the held-out test set, which was strictly reserved for the final assessment of the model’s generalization capabilities on unseen microstructures. This test set was evaluated only once, after training and hyperparameter selection were complete, and the resulting test error is reported together with the validation error in subsection 3.2. This splitting protocol was applied independently to each group of samples defined by its specific resolution and morphological parameters to guarantee that model performance is evaluated fairly across the entire design space.

We now describe the neural network architecture used to map the microstructure to the localization tensor.

Architecture

Given the voxel-based representation of the microstructures and the necessity for translation equivariance, a Convolutional Neural Network (CNN) was selected as the backbone [33]. Specifically, we employ the U-Net architecture [10], which consists of a contracting path (encoder) and an expanding path (decoder). The encoder comprises multiple levels, each containing convolutional layers that capture features across varying frequencies: high-frequency details at higher resolutions and low-frequency features in downsampled representations. This symmetric structure is particularly well-suited for image-to-image regression tasks where input and output share identical spatial dimensions.

Spatial downsampling is performed via 2×2×22 \times2 \times2 convolutions with stride 2. The decoder mirrors this process using transposed convolutions to progressively restore resolution. Crucially, skip connections shuttle feature maps from each encoder level to its counterpart in the decoder, concatenating them before further processing so that high-resolution spatial information directly informs the final prediction.

We implement a parametric U-Net variant designed to facilitate systematic hyperparameter exploration. The architecture exposes several degrees of freedom that jointly control its capacity and computational cost:

  • Number of channels (integer): The base network width, defining the number of feature maps in the initial encoder level. The channels can be seen as the components in the case of tensor fields.

  • Number of levels (integer): The depth of the spatial hierarchy, i.e., the number of distinct resolution scales.

  • Number of layers per level (integer): The number of sequential convolutional blocks allocated to each level.

  • Kernel size (integer): The spatial extent KK of the K×K×KK \times K \times K convolutional windows.

  • Separated convolution (boolean): Whether standard convolutions are factorized into separate spatial and channel-wise operations.

  • Constant channels (boolean): Whether the channel count remains fixed or doubles upon each downsampling step.

  • Bias (boolean): Whether bias terms are included in the linear transformations.

  • Skip connections (categorical): The topology governing how encoder features are routed to the decoder. Options include direct concatenation, no skip connections, or transforming encoder features via an additional convolution operation before passing them to the decoder at constant resolution.

  • Normalization method (categorical): The normalization technique applied after convolution (e.g., batch, instance, disabled).

  • Activation function (categorical): The nonlinear activation used in hidden layers (e.g., LeakyReLU, Sigmoid, or SiLU).

Figure 4 provides a schematic overview of this U-Net, illustrating how the hyperparameters govern the flow from input phase map to output tensor field.

Figure 4. Schematic representation of the parametric U-Net architecture. The diagram illustrates the encoder-decoder structure with multiple levels. Blue arrows indicate convolutional operations, red arrows denote downsampling, green arrows denote upsampling, and gray arrows represent skip connections.

The fundamental building block is a convolutional layer followed by optional normalization and nonlinear activation, as detailed in Figure 5. Each block applies a K×K×KK \times K \times K convolution, an optional bias term, optional normalization N\mathcal{N}, and an activation function ff. The output layer omits the activation so that predictions span the full dynamic range required by the 36 tensor components. All convolutions are implemented with PyTorch [34, 35] Conv3d using circular padding to enforce the periodic boundary conditions inherent to our data.

Figure 5. Schematic of a convolutional block used within the network. The process flow shows the input tensor XX being convolved with weights WW. Subsequent operations include the addition of a bias term bb, normalization N\mathcal{N}, and a nonlinear activation function ff to produce the output tensor YY.

Formally, let X∈RCin×D×D×DX \in\mathbb{R}^{C_{\mathrm{in}} \times D \times D \times D} denote the input tensor, where CinC_{\mathrm{in}} is the number of input channels and DD the spatial resolution. The layer is parameterized by weights W∈RCout×Cin×K×K×KW \in\mathbb{R}^{C_{\mathrm{out}} \times C_{\mathrm{in}} \times K \times K \times K} and biases b∈RCoutb \in\mathbb{R}^{C_{\mathrm{out}}}, where CoutC_{\mathrm{out}} is the number of output channels and KK the (odd) kernel size with half-width κ=(K−1)/2\kappa= (K - 1)/2. Adopting the deep-learning convention of cross-correlation (commonly termed convolution) and enforcing periodic boundaries via the modulo operator, the operation at spatial position (x,y,z)(x, y, z) for output channel coutc_{\mathrm{out}} reads:

Conv3D⁡(X)cout,x,y,z=bcout+∑cin=0Cin−1∑i=0K−1∑j=0K−1∑k=0K−1Wcout,cin,i,j,k⋅Xcin,(x+i−κ) mod D,(y+j−κ) mod D,(z+k−κ) mod D(4)\operatorname{Conv3D}(X)_{c_{\mathrm{out}},x,y,z} = b_{c_{\mathrm{out}}} + \sum_{c_{\mathrm{in}}=0}^{C_{\mathrm{in}}-1} \sum_{i=0}^{K-1} \sum_{j=0}^{K-1} \sum_{k=0}^{K-1} W_{c_{\mathrm{out}},c_{\mathrm{in}},i,j,k} \cdot X_{c_{\mathrm{in}},(x+i-\kappa)\bmod D,(y+j-\kappa)\bmod D,(z+k-\kappa)\bmod D} \tag*{(4)}

The output tensor Y∈RCout×D×D×DY \in\mathbb{R}^{C_{\mathrm{out}} \times D \times D \times D} is then obtained by sequentially applying normalization and activation:

Y=f(N(Conv3D⁡(X)))(5)Y = f\left(\mathcal{N}\left(\operatorname{Conv3D}(X)\right)\right) \tag*{(5)}

The activation functions explored during the hyperparameter tuning are:

LeakyReLU⁡(x)=max⁡(0.01⋅x,x)(6)\operatorname{LeakyReLU}(x) = \max(0.01 \cdot x, x) \tag*{(6)}
Sigmoid⁡(x)=11+e−x(7)\operatorname{Sigmoid}(x) = \frac{1}{1 + e^{-x}} \tag*{(7)}
SiLU⁡(x)=x1+e−x(8)\operatorname{SiLU}(x) = \frac{x}{1 + e^{-x}} \tag*{(8)}

Figure 6 illustrates the architecture of a U-Net configured for 64364^3 inputs, predicting the A1111A_{1111} component of the strain localization tensor. This low-dimensional configuration is used as an illustrative example. The top row shows feature-map evolution, where spatial resolution contracts from 64364^3 to 16316^3 in the encoder, before reversing in the decoder. The bottom row displays learned convolutional kernels at successive depths. Such hierarchical, multiscale feature extraction is essential for resolving both localized strain concentrations and long-range elastic interactions. The small 232^3 kernels are used for downsampling and upsampling using standard and transposed convolutions, respectively, with stride 2. The larger 333^3 kernels are used in the main convolution operations at constant resolution. Hyperparameter values for this illustrative example are listed in Table 1. While neural networks remain largely “black boxes”, this visualization provides intuition on the input-output transformation and illustrates how some hyperparameters can influence performance, computational cost, and memory usage.

Internal structure of a representative U-Net instance

Figure 6. Internal structure of a representative U-Net instance for 64364^{3} inputs. Top row: Feature maps through the U-Net. Bottom row: Learned convolutional kernels for layers 1–5, illustrating the hierarchical extraction of features from fine-scale textures to coarse morphological patterns.

HyperparameterValue
Number of channels2
Number of levels3
Number of layers per level1
Kernel size3
Separated convolutionFalse
Constant channelsTrue
BiasFalse
Skip connectionsDirect
Normalization methodNone
Activation functionSiLU

Table 1. Hyperparameters of the neural network illustrated in Figure 6.

In § 3, these hyperparameters are optimized through random search, but first, the training process is described in the following section.

Training

Loss function

The neural network is optimized following standard deep learning protocols. The training objective is to minimize a scalar loss function that quantifies the discrepancy between the model predictions and the reference data considered as “ground truth”. Gradients of the loss with respect to the learnable parameters are computed via automatic differentiation, facilitating iterative updates through backpropagation.

For this study, we employ the Mean Absolute Error (MAE) as the objective function:

LMAE(Y,Y^)=1B⋅C⋅D3∑b,c,x,y,z∣Y(b,c,x,y,z)−Y^(b,c,x,y,z)∣(9)\mathcal{L}_{\mathrm{MAE}}(Y,\hat{Y}) = \frac{1}{B \cdot C \cdot D^{3}} \sum_{b,c,x,y,z} \left|Y(b,c,x,y,z) - \hat{Y}(b,c,x,y,z)\right| \tag*{(9)}

where BB denotes the batch size, CC represents the number of output components (36 channels), and DD is the spatial resolution. The indices bb, cc, xx, yy, and zz correspond to the batch, component, and spatial coordinates, respectively. Here, YY represents the ground truth tensor field, and Y^\hat{Y} denotes the network prediction.

Because the six macroscopic loadings are unit basis vectors (see subsubsection 2.2.2), each component of Λ\Lambda is dimensionless and the absolute error in Equation (9) is already relative to a unit macroscopic strain. The percentage errors reported throughout this work are therefore obtained by multiplying Equation (9) by 100, i.e., they represent the mean absolute deviation between the predicted and reference localization tensors, expressed in percent. The reported percentage values are absolute deviations of the dimensionless localization tensor, expressed in percent of unit macroscopic strain, not relative errors.

The MAE formulation ensures a spatially uniform penalization of the absolute error, penalizing absolute deviations linearly across all voxels and tensor components.

This loss is minimized using one of the optimization algorithms described in the following section.

Optimizer

The optimization process aims to find the set of neural network parameters θ\theta that minimizes the loss function L\mathcal{L} defined in subsubsection 2.4.1. The choice of optimizer significantly influences the convergence rate, the stability of the training process, and the final generalization error. In the context of predicting physical fields, where the loss landscape can be complex due to the high-dimensional output space (36 components) and the heterogeneous nature of the input microstructures, selecting an appropriate optimization algorithm is critical. We evaluate three distinct optimizers ranging from classical stochastic methods to advanced techniques incorporating matrix orthogonalization.

Stochastic Gradient Descent (SGD) with momentum is a foundational optimization algorithm in deep learning [36]. While standard SGD can suffer from slow convergence and oscillations in directions of high curvature, the addition of a momentum term accumulates a velocity vector in directions of persistent reduction in the loss. This helps the optimizer navigate through ravines in the loss landscape and dampen oscillations. It serves as a robust baseline for comparison, often providing good generalization properties despite slower convergence compared to adaptive methods. The update rule is detailed in Algorithm 1, where lrl_r is the learning rate and β\beta is the momentum coefficient.

Algorithm 1 Stochastic gradient descent with momentum

Hyperparameters: lrl_r, β\beta

Iteration:

Gt=∇θLt(θt−1)G_t = \nabla_{\theta}\mathcal{L}_t(\theta_{t-1})
mt=βmt−1+Gtm_t = \beta m_{t-1} + G_t
θt=θt−1−lrmt\theta_t = \theta_{t-1} - l_r m_t

AdamW [37] is a variant of the Adam optimizer [38] that implements decoupled weight decay regularization. Adam adapts the learning rate for each parameter based on the first and second moments of the gradients, allowing for efficient navigation of sparse gradients and noisy objectives. However, standard Adam couples L2 regularization with the gradient update, which can lead to suboptimal regularization behavior. AdamW decouples this, applying weight decay λ\lambda directly to the weights rather than the gradient. This method is widely adopted in modern deep learning architectures due to its fast convergence and stability. The specific update steps, including bias correction for the moment estimates (m^t,v^t)(\hat{m}_t,\hat{v}_t), are presented in Algorithm 2.

Algorithm 2 AdamW

Hyperparameters: lrl_r, β1\beta_1, β2\beta_2, λ\lambda, ϵ\epsilon

Iteration:

Gt=∇θLt(θt−1)G_t = \nabla_{\theta} \mathcal{L}_t(\theta_{t-1})
mt=β1mt−1+(1−β1)Gtm_t = \beta_1 m_{t-1} + (1-\beta_1)G_t
vt=β2vt−1+(1−β2)Gt2v_t = \beta_2 v_{t-1} + (1-\beta_2)G_t^2
m^t=mt1−β1t,v^t=vt1−β2t\hat{m}_t = \frac{m_t}{1-\beta_1^t}, \qquad\hat{v}_t = \frac{v_t}{1-\beta_2^t}
θt=θt−1−lr(m^tv^t+ϵ+λθt−1)\theta_t = \theta_{t-1} - l_r \left(\frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}+\lambda\theta_{t-1}\right)

The Muon optimizer [39] represents a more recent class of optimization algorithms that incorporate second-order information or matrix structural constraints to improve stability. As illustrated in Algorithm 3, Muon computes the momentum buffer similarly to SGD but applies a Newton-Schulz (NS) iteration, denoted as NS(⋅\cdot), to the momentum matrix mtm_t before the parameter update. This orthogonalization step aims to stabilize the update direction, potentially preventing the exploding or vanishing gradient problems that can occur in deep networks with complex physical constraints. By enforcing orthogonality on the update matrix, Muon seeks to maintain a well-conditioned optimization trajectory, which may be particularly beneficial for the high-dimensional tensor field predictions targeted in this study. In practice, we employ a hybrid optimization strategy to accommodate the structural requirements of the Muon algorithm. Muon is exclusively applied to the convolutional kernels of the intermediate hidden layers, whereas AdamW is utilized for all other parameters, including the input and output layer weights and any optional bias terms. Since the convolutional kernels defined in Equation 4 are 5D tensors (Cout×Cin×K×K×KC_{\mathrm{out}} \times C_{\mathrm{in}} \times K \times K \times K), they must be reshaped to facilitate the matrix orthogonalization inherent to the Newton-Schulz iteration. Specifically, the last four dimensions are flattened to convert the update tensors into matrices before orthogonalization, and folded back into their original shape before the parameter update. This procedure ensures compatibility with the convolutional operation while benefiting from the stability of orthogonal updates.

Algorithm 3 Muon

Hyperparameters: lrl_r, β\beta

Iteration:

Gt=∇θLt(θt−1)G_t = \nabla_{\theta} \mathcal{L}_t(\theta_{t-1})
mt=βmt−1+Gtm_t = \beta m_{t-1} + G_t
Ot=NS⁡(mt)O_t = \operatorname{NS}(m_t)
θt=θt−1−lrOt\theta_t = \theta_{t-1} - l_r O_t

Results

Hyperparameter tuning

To achieve an optimal trade-off between predictive accuracy and inference latency, a systematic hyperparameter optimization strategy was employed. The optimal configuration was identified through three sequential search stages, designed to progressively refine the search space. The initial search encompassed a broad parameter space with approximately 700 million possible combinations. Based on the validation MAE obtained from this coarse search, the parameter ranges were subsequently narrowed by excluding underperforming regions and fixing insensitive hyperparameters to values that minimize inference cost. Each of the three search stages comprised 1370 trials, where each trial involved training a model from scratch with random hyperparameter values sampled within the ranges defined in Table 2. Search 1 was performed only for the baseline configuration, whereas Search 2 and Search 3 were performed independently for the baseline and the three scaling configurations, i.e., 4 × 1370 trials per stage. To maintain computational feasibility, each trial was subject to a time-constrained training budget of 300 s (Search 1 and Search 2) and 600 s (Search 3) on a single Nvidia H100 GPU. Configurations deemed infeasible due to algorithmic constraints (e.g., convolution folding resulting from large kernels combined with excessive depth) or memory limitations (out-of-memory errors) were pruned from the search space.

Table: Evolution of the hyperparameter search space across three sequential refinement stages, transitioning from coarse exploration (Search 1) to constrained tuning (Search 2–3). The ranges in each column apply to each configuration. Search 1 covers the baseline configuration only; Search 2 and Search 3 cover the baseline and the three scaling configurations.

Search 1Search 2Search 3
Architecture hyperparameters
Number of channels[2, 4, 8, 12, 16, 24, 32, 48, 64, 96, 128][2, 4, 8, 12, 16, 24, 32, 48, 64, 96, 128][8, 12, 16, 24, 32, 48, 64, 96, 128]
Number of levels[1, 2, 3, 4, 5, 6, 7, 8][3, 4, 5, 6, 7, 8][3, 4, 5, 6, 7, 8]
Number of layers per level[1, 2, 3, 4]11
Kernel size[2, 3, 4, 5, 6, 7]33
Separated convolution[False, True]FalseFalse
Constant channels[False, True]TrueTrue
Bias[False, True]FalseFalse
Skip connections[off, direct, trainable]directdirect
Normalization method[off, instance, batch][off, instance, batch]off
Activation function[LeakyReLU, Sigmoid, SiLU][LeakyReLU, Sigmoid, SiLU]SiLU
Training hyperparameters
Optimizer[SGD, AdamW, Muon]MuonMuon
Batch size[1, 2, 4, 8, 16]11
Learning rate[10−610^{-6}, 10010^{0}][10−610^{-6}, 10010^{0}][10−510^{-5}, 10−110^{-1}]
Momentum[0.5, 0.7, 0.9, 0.95, 0.99, 0.995, 0.999, 0.9999][0.5, 0.7, 0.9, 0.95, 0.99, 0.995, 0.999, 0.9999][0.9, 0.95, 0.99, 0.995, 0.999]
Training budget300 s300 s600 s

Table 2. Evolution of the hyperparameter search space across three sequential refinement stages, transitioning from coarse exploration (Search 1) to constrained tuning (Search 2–3). The ranges in each column apply to each configuration. Search 1 covers the baseline configuration only; Search 2 and Search 3 cover the baseline and the three scaling configurations.

To mitigate the computational cost associated with optimizing hyperparameters for the full-scale problem, a progressive scaling study was conducted. The optimization process began at a low-complexity baseline: a resolution of 64364^{3}, predicting only the first 6 components of the localization tensor, and utilizing a single type of microstructure morphology (denoted as the baseline configuration in Figure 7). Subsequently, the complexity was incrementally increased along three independent axes: spatial resolution, output dimensionality (number of tensor components), and morphological diversity. This approach, illustrated by the intermediate configurations in Figure 7, allowed for the evaluation of scaling laws to determine which optimal hyperparameters remain invariant and which require adjustment as problem complexity increases. These scaling rules were then applied to define the final training configuration for the high-complexity target scenario (resolution 2563256^{3}, 36 components, and varied morphologies), represented by the target configuration in Figure 7.

Figure 7. Schematic of the progressive scaling study mapping the trajectory from low-complexity baseline to high-complexity target across spatial resolution, output dimensionality, and morphological diversity. The black dot represents the baseline configuration (64364^3, 6 components, 1 morphology), while the gray dot indicates the target configuration (2563256^3, 36 components, 4 morphologies). The colored dots (blue, green, red) represent intermediate configurations used to assess scaling behavior.

The hyperparameter tuning study was performed using Optuna [40] and MLflow [41].

The results of the hyperparameter optimization for the baseline configuration (64364^3 resolution, 6 output components, and single morphology, Search 3) are presented in Figure 8. Due to the extensive search space, we focus on three key hyperparameters for this baseline configuration to illustrate the general trends obtained; the complete set of graphs for all searches is provided in the supplementary material. In the figures, triangles indicate the 50th percentile (median), squares represent the 10th percentile, and circles denote the minimum values observed across trials. These minimum values are particularly significant as they represent the best performance achievable by a model configuration for each specific hyperparameter value. The analysis evaluates the trade-off between predictive accuracy, measured by the Mean Absolute Error (MAE), and computational efficiency, measured by the inference time per sample.

Figure 8. Hyperparameter tuning results for Search 3 of the baseline configuration (64364^3 resolution, 6 output components, 1 morphology). (a, c, e) show the Mean Absolute Error (MAE), while (b, d, f) show the inference time per sample. The plots illustrate the sensitivity of model performance to the number of channels, number of levels, and learning rate exponent. The violin plots are obtained using Seaborn [42].

Figure 8 panels (a) and (b) illustrate the impact of the number of channels. A clear decrease in MAE is observed up to 64 channels, beyond which only marginal gains are achieved. Regarding computational cost, inference time increases monotonically with the number of channels. Consequently, the number of channels serves as an effective hyperparameter to balance accuracy against speed.

The influence of network depth, represented by the number of levels, is shown in Figure 8(c, d). A clear trend emerges where increasing the number of levels from 3 to 5 consistently reduces the MAE, with marginal gains observed above 5 levels. Notably, this improvement in accuracy is achieved with minimal impact on inference time, which remains relatively stable across different depths. At this resolution, 5 levels represent the ideal balance, indicating that increasing the depth of the U-Net is an efficient strategy for enhancing performance without significantly compromising speed.

Finally, the sensitivity to the learning rate is depicted in Figure 8(e, f). The MAE exhibits a strong dependence on the learning rate exponent. Values that are too small (lr≤10−4.5l_r \le10^{-4.5}) result in slower convergence and higher error, while values that are too large (lr≥10−1.5l_r \ge10^{-1.5}) lead to instability and a drastic increase in error. The optimal range is found between learning rates 10−4.010^{-4.0} and 10−2.010^{-2.0}, where the model achieves the best compromise between accuracy and stability. Inference time remains largely unaffected by the learning rate, as expected, since the learning rate governs the training dynamics rather than the computational graph used during inference.

The optimal hyperparameters presented in Table 3 and Table 4 were selected from the full set of trials in Search 3 across each configuration. The selection followed two distinct criteria to address different application needs:

r64-c6-m1r128-c6-m1r64-c36-m1r64-c6-m4
Architecture hyperparameters
Number of channels12864128128
Number of levels7775
Training hyperparameters
Learning rate10−310^{-3}10−310^{-3}3⋅10−33 \cdot10^{-3}10−310^{-3}
Momentum0.9990.9950.990.999
Metrics
Mean Absolute Error0.19 %0.35 %0.38 %0.38 %
Inference time per sample5.142 ms19.355 ms7.070 ms5.037 ms

Table 3. Accuracy-optimal hyperparameters for different configurations.

r64-c6-m1r128-c6-m1r64-c36-m1r64-c6-m4
Architecture hyperparameters
Number of channels882416
Number of levels5645
Training hyperparameters
Learning rate3⋅10−43 \cdot10^{-4}10−310^{-3}3⋅10−33 \cdot10^{-3}10−310^{-3}
Momentum0.950.990.9990.995
Metrics
Mean Absolute Error0.95 %0.97 %0.98 %0.93 %
Inference time per sample0.779 ms5.385 ms3.033 ms1.033 ms

Table 4. Speed-optimal hyperparameters for different configurations.

  1. Accuracy-optimal: the configuration yielding the minimum validation MAE, regardless of inference time.

  1. Speed-optimal: the configuration yielding the minimum inference time, subject to the constraint that the validation MAE remains lower than 1 %.

This dual-criterion approach allows users to tailor the surrogate model based on whether the priority is maximum fidelity or computational throughput.

Analysis of the accuracy-optimal configurations in Table 3 reveals several trends regarding the scaling of network capacity with problem complexity. For the baseline configuration (r64-c6-m1), the optimal model utilizes 128 channels and 7 levels, achieving a remarkably low error of 0.19 %. When the resolution is increased to 1283128^3 (r128-c6-m1), the optimal number of channels decreases to 64, while the depth remains at 7 levels. Investigation of the 128-channel trials reveals performance metrics very close to those of the 64-channel trials, corroborating the finding discussed in subsection 3.1 that accuracy gains are marginal above 64 channels. Increasing the output dimensionality from 6 to 36 components (r64-c36-m1) requires maintaining the high channel count (128) but results in a slightly higher error (0.38 %), indicating the increased difficulty of mapping multiple tensor components simultaneously. Similarly, increasing morphological diversity from 1 to 4 configurations (r64-c6-m4) reduces the optimal depth to 5 levels. Investigation shows that 5 or more levels yield similar results, consistent with the earlier observation that marginal gains are achieved above 5 levels.

The speed-optimal configurations in Table 4 demonstrate a significant reduction in model capacity to meet the inference time constraints. The number of channels drops drastically to between 8 and 24, and the number of levels is reduced to between 4 and 6. Despite this reduction, the error remains bounded just below the 1 % threshold (ranging from 0.93 % to 0.98 %). The inference time improvements are substantial; for the baseline configuration, the speed-optimized model is approximately 6.6 times faster than the accuracy-optimized model (0.779 ms vs 5.142 ms), highlighting the efficiency gains achievable when strict accuracy requirements are relaxed slightly.

In addition to the primary architectural dimensions of width and depth, we systematically evaluate several secondary hyperparameters to identify configurations that balance performance with computational efficiency. Regarding convolutional geometry, small odd-sized kernels (e.g., K=3K = 3) consistently outperform larger or even-sized kernels, whereas separated convolutional layers yield suboptimal performance. Structural choices also influenced efficiency: maintaining a constant channel count across levels yielded accuracy comparable to doubling channels upon downsampling but with a reduced memory footprint. Bias terms were excluded as they provided no measurable improvement, and direct skip connections proved superior to both absent and trainable skip connection topologies. Normalization layers (batch or instance) offered negligible benefits over no normalization, favoring the latter for computational simplicity. Among activation functions, SiLU yielded marginally lower errors than LeakyReLU, with Sigmoid performing worst. Regarding optimization, the Muon optimizer demonstrated superior performance relative to AdamW and SGD with momentum, characterized by enhanced stability in learning curves throughout training iterations and reduced final validation errors. A monotonic increase in validation error was observed with increasing batch sizes. Finally, optimal learning rate and momentum values were identified within specific ranges and remained invariant despite increases in problem complexity.

In conclusion, this analysis demonstrates that hyperparameters can be effectively optimized for either accuracy or speed based on specific user requirements. For a fixed objective, most optimal hyperparameters remain stable when scaling problem complexity (resolution, components, morphologies). Only two parameters require significant adjustment:

  1. The number of channels should be increased or decreased to control the accuracy versus speed tradeoff.

  1. The number of levels should increase with the resolution such that the minimum resolution of the feature maps (at the end of the encoder) reaches 434^3.

Applying these guidelines, we now train a single model on the full target problem.

Final training run

The final training run targets the high-complexity configuration identified in Figure 7, namely a spatial resolution of 2563256^3 voxels, the full 36-component localization tensor output, and the full morphological diversity comprising the four aggregate volume fraction and aspect ratio combinations described in subsubsection 2.2.1.

Based on the hyperparameter tuning analysis presented in subsection 3.1, a final configuration was selected to balance the competing objectives of predictive accuracy and inference speed. The chosen compromise utilizes 48 channels to balance accuracy and speed, and 7 levels leading to a minimum feature map resolution of 434^3. The complete set of final hyperparameters is detailed in Table 5.

HyperparameterValue
Number of channels48
Number of levels7
Number of layers per level1
Kernel size3
Separated convolutionFalse
Constant channelsTrue
BiasFalse
Skip connectionsdirect
Normalization methodoff
Activation functionSiLU
OptimizerMuon
Batch size1
Learning rate10−310^{-3}
Momentum0.999

Table 5. Final hyperparameters selected for the target configuration: 2563256^3 resolution, 36 localization-tensor components, four morphologies.

The selected architecture contains approximately 1.4 million trainable parameters. Peak GPU memory consumption during training reached approximately 90 GB, which remains within the capacity of a single Nvidia H100 GPU but illustrates the substantial memory requirements of high-resolution 3D image-to-image regression.

The training dynamics for this configuration are illustrated in Figure 9. The model demonstrates rapid initial convergence, with the Mean Absolute Error decreasing below 2 % within approximately 100 epochs, corresponding to a training time of roughly 35 min. Following this initial phase, the training and validation errors begin to separate slightly. The training process was terminated once the validation error had stagnated, after a total training time of 240 min, reaching a final training error of 0.95 %, a validation error of 1.1180 %, and a test error of 1.1184 %. The test error was computed once on the held-out samples, after training and model selection were complete; its agreement with the validation error to within 0.0004 percentage points indicates that the validation-based stopping and hyperparameter choices did not materially bias the reported performance.

Figure 9. Learning curves showing the evolution of training and validation errors over epochs.

A qualitative assessment of the model’s performance is provided in Figure 10, which compares the first component of the target strain localization field A1111targetA^{\mathrm{target}}_{1111} against the neural network prediction A1111predictionA^{\mathrm{prediction}}_{1111} for a representative validation sample. Visually, the target and prediction maps appear highly similar, capturing the complex heterogeneity of the strain field. The difference map highlights the discrepancy between the two, revealing that the highest errors are concentrated at the interfaces between the aggregates and the cement matrix. This is physically consistent, as these regions exhibit the steepest gradients in mechanical properties and strain concentrations, making them the most challenging features to resolve. Because the reported MAE is averaged over all voxels and components, these local discrepancies are not visible in that scalar: errors in the interfacial regions are larger than the global average, which should therefore not be read as a per-voxel accuracy guarantee.

Validation sample comparison

Figure 10. Validation sample comparison. (a) Target strain localization field (considered as “ground truth”), (b) Neural network prediction, and (c) Difference between the two.

The scalar MAE of Equation (9) averages over all 36 components. To verify that the aggregate value does not conceal localized inaccuracies on individual components, Equation (10) reports the test MAE separately for each component, computed on the held-out test set and arranged in the same 6×66 \times6 Voigt-Mandel form as Figure 3. Each entry MijMAEM^{\mathrm{MAE}}_{ij} is the test MAE (in percent) for the component AijA_{ij}, where the row index ii denotes the microscopic strain component and the column index jj the macroscopic loading component, both ordered as (11,22,33,23,13,12)(11, 22, 33, 23, 13, 12):

MMAE=[1.470.930.920.991.211.240.901.500.931.230.961.180.920.941.541.251.191.000.871.041.041.471.121.091.070.891.041.091.471.111.061.060.881.091.111.45](10)M^{\mathrm{MAE}} = \begin{bmatrix} 1.47 & 0.93 & 0.92 & 0.99 & 1.21 & 1.24 \\ 0.90 & 1.50 & 0.93 & 1.23 & 0.96 & 1.18 \\ 0.92 & 0.94 & 1.54 & 1.25 & 1.19 & 1.00 \\ 0.87 & 1.04 & 1.04 & 1.47 & 1.12 & 1.09 \\ 1.07 & 0.89 & 1.04 & 1.09 & 1.47 & 1.11 \\ 1.06 & 1.06 & 0.88 & 1.09 & 1.11 & 1.45 \end{bmatrix} \tag*{(10)}

All component errors lie between 0.87% and 1.54%, with the largest error on the normal A3333A_{3333} component and the largest coupling error on the A3323A_{3323} term (1.25%). These component-level values are consistent with the overall test MAE of 1.1184% reported above, so the aggregate value does not conceal any component-level outlier.

Finally, the computational efficiency of the trained surrogate model is compared with that of the FFT-based solver in Table 6. Both methods were run on an Nvidia H100 GPU in single precision (float32), with a batch size of one and PyTorch JIT compilation. The FFT timing corresponds to the complete localization tensor, i.e., the six unit loadings of sub subsection 2.2, so that both methods produce the full 36-component field. Under these conditions, the solver required 1.39 s per sample at a mean power consumption of 450 W, whereas a single forward pass of the network completed in 0.16 s, corresponding to a speedup of approximately 8.7. Although the network draws more instantaneous power (600 W), likely because the tensor cores are heavily used during inference, its shorter runtime lowers the energy cost per prediction from 625 J to 96 J, a reduction by a factor of approximately 6.5. This acceleration confirms the suitability of the proposed surrogate modeling approach for multiscale simulations where numerous microstructural computations are required. This comparison is per prediction and does not include the one-off costs of dataset generation and hyperparameter optimization; the surrogate therefore becomes advantageous only when the number of RVE evaluations is large, as in the coupled multiscale simulations targeted here.

HardwareMethodPower consumptionComputation timeEnergy per sample
GPU H100FFT-based solver450 W1.39 s625 J
GPU H100Neural network600 W0.16 s96 J

Table 6. Comparison of computational cost and power consumption between the FFT-based solver and the trained neural network. Both methods use float32, a batch size of one, and PyTorch JIT compilation on an Nvidia H100 GPU.

While speed and accuracy are essential, practical deployment also requires an understanding of how the surrogate generalizes beyond its training distribution.

Generalization

Despite their computational advantages, neural network surrogates are inherently constrained by their training distribution: they are reliable only when applied to microstructures that are statistically similar to those seen during training. To quantify this limitation in the context of concrete mesostructures, we assess whether a U-Net trained on one morphology can generalize to unseen aggregate volume fractions and aspect ratios. In addition to direct error measurements, we introduce a Maximum Mean Discrepancy (MMD) metric, computed on the model’s internal feature maps, which provides a diagnostic tool to decide whether a new microstructure lies within the training distribution without requiring the FFT reference computation.

To assess the robustness of the proposed framework against morphological variations, a cross-evaluation study was conducted using the optimal hyperparameters identified in subsection 3.1. Five distinct neural network instances were trained from scratch, categorized into two groups based on the diversity of their training data. The first group comprised four specialized models (A–D), each trained exclusively on a single morphological configuration defined by specific aggregate volume fractions and aspect ratios (see Figure 2). Models A and B corresponded to a 20 % volume fraction with aspect ratios of 100 % and 150 %, respectively, while Models C and D corresponded to a 40 % volume fraction with the same aspect ratios. The second group consisted of a single generalist model (Model E), trained on the aggregated dataset encompassing all four morphological configurations.

All generalization experiments were conducted at a resolution of 64364^3 voxels. This lower resolution was chosen to enable the rapid training of multiple independent models from scratch while preserving the essential morphological features required to assess cross-configuration generalization. All five models were trained with identical hyperparameters, using a compromise configuration adapted to 64364^3 resolution.

Mean absolute error

To evaluate generalization capabilities, all five models were subsequently tested against the full suite of four morphologies. This experimental design facilitated a direct comparison between in-distribution performance and out-of-distribution generalization. Specifically, it quantifies the performance degradation incurred when a specialized model (e.g., Model A) is applied to unseen microstructural geometries (e.g., the morphology of Model D), relative to the baseline performance of the generalist Model E across all configurations.

The five models described above were trained with a convergence criterion set at a Mean Absolute Error (MAE) of 1 %. As indicated by the step counts in Figure 11, the computational cost to reach this threshold varied significantly by morphology, ranging from 3264 steps for the f20–s100 configuration to 9984 steps for the aggregated dataset. Specialized models exhibit strong in-distribution performance, with validation errors on their respective training morphologies aligning closely with the 1 % stopping threshold. However, out-of-distribution testing reveals significant limitations in generalization, characterized by a marked asymmetry. Models trained on low volume fraction morphologies (VF = 20 %) show performance degradation when applied to high volume fraction configurations (VF=40\mathrm{VF} = 40%), with errors ranging between 3.07% and 4.43%. Conversely, models trained on VF=40\mathrm{VF} = 40% data generalize more effectively to VF=20\mathrm{VF} = 20% validation cases, maintaining errors below 1.5%. The generalist model (labeled All), trained on the full dataset, demonstrates superior robustness. Despite requiring the longest training duration, it achieves the lowest error rates across all validation configurations, consistently maintaining an MAE below 1% (ranging from 0.52% to 0.98%).

Figure 11. Generalization performance matrix showing Mean Absolute Error (MAE) for specialized and generalist models. Rows denote training configurations and columns denote validation configurations. Training was terminated upon reaching a 1% MAE threshold on the training set (step counts indicated on the left). Specialized models display significant error escalation when tested on morphologies with higher volume fractions than those seen during training. In contrast, the generalist model (All), trained on the aggregated dataset, exhibits consistent high accuracy across all morphological variations, outperforming specialized models in out-of-distribution scenarios.

Maximum mean discrepancy

Maximum Mean Discrepancy (MMD), a kernel-based two-sample statistic widely used for distribution-shift quantification [43], has been employed in microstructure-informed deep learning to assess surrogate prediction reliability by quantifying distributional shifts in learned feature representations [44]. Following this approach, MMD is computed in the learned feature space of each U-Net model to quantify the discrepancy between validation microstructures and the training distribution.

Features are extracted from the encoder_3_conv_0 layer, which provides a deep representation of morphology-relevant and prediction-relevant features. MMD computed from this representation consistently exhibits a strong correlation with prediction error.

MMD is evaluated using a Radial Basis Function (RBF) kernel, with the kernel bandwidth determined from the reference features using the median pairwise-distance heuristic. For each specialized model, the 192 training microstructures are divided into 160 reference samples and 32 held-out in-distribution samples. The reference samples define the latent training distribution, while the held-out samples establish an independent in-distribution baseline MMD. Each validation microstructure is compared individually against the reference distribution, and the MMD values are averaged over the 32 microstructures in each validation dataset. For the generalist model, the reference distribution consists of 160 samples񎟙 from each of the four training classes (640 samples total), and the baseline is obtained by averaging the class-specific baseline MMDs.

The distribution shift is expressed relative to the in-distribution baseline as

MMDrel=MMDval−MMDbaselineMMDbaseline.(11)\mathrm{MMD}_{\mathrm{rel}} = \frac{\mathrm{MMD}_{\mathrm{val}} - \mathrm{MMD}_{\mathrm{baseline}}}{\mathrm{MMD}_{\mathrm{baseline}}}. \tag*{(11)}

A relative MMD near zero indicates a distribution comparable to the in-distribution baseline, while increasing positive values indicate greater departure from the training distribution. Negative values indicate a discrepancy below the estimated in-distribution baseline and are therefore also interpreted as being within the baseline distributional variability. Because each surrogate learns a distinct latent representation and kernel bandwidth, relative MMD values are interpreted primarily within each model rather than quantitatively across models.

Figure 12 reports the relative MMD, expressed as the percentage change with respect to each model’s in-distribution baseline MMD. For each specialized model, the baseline MMD was calculated using a held-out subset of the training data. The validation microstructures were then compared against the corresponding training distribution. Since each model learns a distinct latent representation, the MMD values should be interpreted primarily within rows rather than compared quantitatively across models.

Figure 12. Mean relative Maximum Mean Discrepancy (MMD) between the latent representations of validation microstructures and the corresponding training reference distributions. Values represent the percentage change relative to the in-distribution baseline MMD for each model. Each row should be interpreted independently because every model learns a distinct latent representation. Green and red shading correspond to small and large latent distribution shifts, respectively. The encoder is more sensitive to aggregate volume fraction than to aggregate shape, and the generalist model (All) exhibits uniformly low MMD across all validation configurations.

The specialized models exhibit progressively larger relative MMD as the validation microstructures deviate from their corresponding training morphology. Changes in aggregate volume fraction consistently produce substantially larger latent distribution shifts than changes in aggregate shape. In contrast, the generalist model, trained on the combined dataset, exhibits small relative MMD variations across all four microstructure classes, indicating broader coverage of the microstructure design space.

Although MMD does not fully explain why Models C and D generalize relatively well to lower-volume-fraction microstructures, a large MMD reliably signals out-of-distribution inputs and thus serves as a practical, computationally inexpensive indicator of surrogate trustworthiness.

Table 7 reports the Pearson correlation coefficient (rr) between the relative MMD and the corresponding prediction error (MAE) across the four validation datasets for each specialized model. The consistently high positive correlations indicate that increasing latent distribution shift is associated with increasing prediction error, supporting MMD as an effective indicator of distribution shift, model generalization, and prediction reliability for previously unseen microstructure datasets. A Pearson correlation is not reported for the generalist model because all evaluation datasets are represented in its training distribution, and the remaining prediction errors are no longer primarily governed by distribution shift.

ModelCorrelation rr
Model A0.98
Model B0.99
Model C0.88
Model D0.86

Table 7. Pearson correlation coefficient representing the linear correlation between the MAE presented in Figure 11 and the MMD presented in Figure 12.

Discussion

Beyond the quantitative results, several design and deployment implications emerge from this study.

A key contribution of this work is the comprehensive hyperparameter optimization framework, which identified robust design rules for adapting the U-Net architecture to increasing problem complexity. Our investigation has revealed that most architectural and training hyperparameters can be fixed across resolutions, output dimensionalities, and morphological diversities, with only two parameters requiring careful scaling: the number of channels, which controls the accuracy-speed trade-off, and the number of levels, which should be increased with resolution to maintain a minimal encoder feature-map resolution of 434^3. Secondary design choices, including 333^3 non-separated convolutions, direct skip connections, omitted normalization and bias terms, and the SiLU activation function, were found to yield the best compromise between accuracy and computational efficiency. Notably, maintaining a constant channel depth across encoder levels, rather than doubling the number of channels after each downsampling step as is conventional in most U-Net implementations, was found to deliver comparable predictive accuracy while significantly reducing memory consumption. This departure from the standard U-Net convention is particularly important for 3D microstructures, where memory constraints rapidly become the limiting factor.

The choice of optimizer also proved influential, with the Muon algorithm consistently outperforming AdamW and SGD with momentum for this high-dimensional image-to-image regression task. To the best of our knowledge, this work constitutes the first application of the Muon optimizer to a continuum solid mechanics problem. Muon has recently become the de facto optimizer for the pretraining of several frontier large language models [39, 45], where the Newton-Schulz orthogonalization of the update matrix is valued for its training stability and efficiency. The present results suggest that this stabilization is not specific to language modeling, but extends to the high-dimensional, spatially structured tensor-field regression problems encountered in computational homogenization.

The most important practical takeaway is that the “optimal” model configuration is not universal: it critically depends on the user’s objectives. The dual-criterion selection strategy proposed in this study distinguishes between accuracy-optimal and speed-optimal configurations. Accuracy-optimal models achieve validation errors below 0.4 % but require larger networks with inference times exceeding 5 ms per sample, whereas speed-optimal models reduce inference time by up to a factor of six while retaining errors below the 1 % threshold. This underscores the necessity of clearly defining application-specific goals before hyperparameter optimization: a surrogate intended for offline homogenization of a limited set of microstructures may prioritize accuracy, whereas one embedded in a coupled multiscale simulation with millions of RVE evaluations may prioritize throughput.

Closely related to this objective-dependent optimum is the influence of the stopping criterion. The hyperparameter search employed time-constrained training budgets, and the final model was trained until validation error stagnated. Different training durations or convergence thresholds would naturally shift the accuracy-speed trade-off analysis. For instance, shorter budgets favor smaller, faster-converging networks, while extended training can unlock additional accuracy from larger architectures. Therefore, the reported optimal hyperparameters should be understood in the context of the specific computational budget and error tolerance under which they were obtained. Researchers and practitioners should align the optimization objective and stopping criterion with the operational constraints of their target multiscale framework.

Generalization experiments revealed important limitations inherent to data-driven surrogates. Specialized models trained on a single morphology exhibited strong in-distribution performance but significant degradation when applied to microstructures with higher aggregate volume fractions than those seen during training. A generalist model trained on a diverse ensemble of morphologies consistently outperformed specialized models, maintaining validation errors below 1 % across all configurations. This highlights the value of training-data diversity when the surrogate is expected to handle microstructural variability. To mitigate reliability concerns outside the training distribution, we proposed the use of a Maximum Mean Discrepancy metric computed on internal feature maps as a diagnostic indicator. Such a metric can flag out-of-distribution inputs without requiring costly reference computations, thereby enabling informed decisions about surrogate trustworthiness in operational settings.

While this study establishes a robust framework for linear elastic regimes, several assumptions present fertile ground for future investigation. First, the current methodology is restricted to linear elasticity; extending it to inelastic behavior, such as plasticity or damage evolution, would require accounting for loading history. One route is to treat the loading sequence as an additional input dimension, yielding a 4D input domain (3D structure + 1D loading sequence). Although computationally demanding, recent advances in sequence modeling for mechanics [14] suggest that this route is conceptually feasible. Second, the scalability of the proposed architecture warrants further exploration regarding domain size transferability. Specifically, future work could investigate whether a U-Net trained on smaller representative volume elements can reliably predict fields in significantly larger domains, potentially offering a strategy to reduce the computational burden associated with generating high-resolution training data.

Conclusion

This study has presented a methodology for developing U-Net surrogate models to predict the full 36-component strain localization tensor field in 3D voxelized concrete mesostructures. By framing the mechanical problem in terms of the localization tensor, the trained surrogate enables the reconstruction of microscopic strain and stress fields for arbitrary macroscopic loadings, offering an efficient alternative to traditional solvers. The hyperparameter study further showed that only network width and depth need to be scaled with problem complexity, while most other design choices can remain fixed.

The final trained surrogate achieved a test MAE of 1.118 %, essentially identical to the validation MAE, and an inference speedup of approximately 8.7×8.7 \times for the full 36-component tensor compared to the FFT-based solver on an Nvidia H100 GPU, with a corresponding reduction in total energy consumption. The successful scaling of the U-Net surrogate to high-resolution three-dimensional microstructures represents the principal novelty of this work. Extending deep-learning-based field prediction to 2563256^3 voxel domains is challenging because of the substantial memory requirements associated with volumetric feature maps and the computational burden of regressing high-dimensional tensor fields. Nevertheless, the optimized architecture and training pipeline proved capable of handling these large 3D volumes, delivering predictions faster than an FFT-based solver, itself already recognized as a highly efficient numerical method for periodic heterogeneous media. Qualitative assessment confirmed that the largest prediction errors are concentrated at aggregate-matrix interfaces, which is physically consistent with the steep property gradients and strain concentrations present in these regions.

Taken together, this work demonstrates that carefully optimized U-Net surrogates can deliver accurate and computationally efficient predictions of strain localization fields in heterogeneous materials, provided that the model configuration is tailored to the specific accuracy, speed, and generalization requirements of the intended application. The clear definition of optimization objectives and stopping criteria is essential to navigating the inherent trade-offs and unlocking the full potential of machine learning surrogates in multiscale mechanical simulations.

Acknowledgments

The authors gratefully acknowledge the financial support of the Swiss Federal Office of Energy (Contract No. SI/502787-01) and the Swiss National Science Foundation (Grant No. 221743). We also thank Somdatta Goswami and Nicolas Richart for their helpful discussions and constructive feedback.

References

  1. [1]H. Moulinec and P. Suquet, “A fast numerical method for computing the linear and nonlinear mechanical properties of composites,” Comptes rendus de l’Académie des sciences. Série II. Mécanique, physique, chimie, astronomie, pp. 1417–1423, 1994.
  2. [2]H. Moulinec and P. Suquet, “A numerical method for computing the overall response of nonlinear composites with complex microstructure,” Computer Methods in Applied Mechanics and Engineering, vol. 157, pp. 69–94, Apr. 1998.DOI
  3. [3]F. Feyel and J.-L. Chaboche, “FE2 multiscale approach for modelling the elastoviscoplastic behaviour of long fibre SiC/Ti composite materials,” Computer Methods in Applied Mechanics and Engineering, vol. 183, pp. 309–330, Mar. 2000.DOI
  4. [4]J. Spahn, H. Andrä, M. Kabel, and R. Müller, “A multiscale approach for modeling progressive damage of composite materials using fast Fourier transforms,” Computer Methods in Applied Mechanics and Engineering, vol. 268, pp. 871–883, Jan. 2014.DOI
  5. [5]D. Bishara, Y. Xie, W. K. Liu, and S. Li, “A State-of-the-Art Review on Machine Learning-Based Multiscale Modeling, Simulation, Homogenization and Design of Materials,” Archives of Computational Methods in Engineering, vol. 30, pp. 191–222, Jan. 2023.DOI
  6. [6]Y. Wang, K. Wang, and C. Zhang, “Applications of artificial intelligence/machine learning to high-performance composites,” Composites Part B: Engineering, vol. 285, p. 111740, Oct. 2024.DOI
  7. [7]R. Liu, Y. C. Yabansu, A. Agrawal, S. R. Kalidindi, and A. N. Choudhary, “Machine learning approaches for elastic localization linkages in high-contrast composite materials,” Integrating Materials and Manufacturing Innovation, vol. 4, pp. 192–208, Dec. 2015.DOI
  8. [8]Z. Yang, Y. C. Yabansu, D. Jha, W.-k. Liao, A. N. Choudhary, S. R. Kalidindi, and A. Agrawal, “Establishing structure-property localization linkages for elastic deformation of three-dimensional high contrast composites using deep learning approaches,” Acta Materialia, vol. 166, pp. 335–345, Mar. 2019.DOI
  9. [9]P. M. Khanolkar, C. C. McComb, and S. Basu, “Predicting elastic strain fields in defective microstructures using image collection algorithms,” Computational Materials Science, vol. 186, p. 110068, Jan. 2021.DOI
  10. [10]O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Medical Image Computing and Computer-Assisted Intervention (N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds.), vol. 9351, (Cham), pp. 234–241, Springer International Publishing, 2015.arxiv.org/abs/1505.04597
  11. [11]M. Raj, S. Thakre, R. K. Annabattula, and A. K. Kanjarla, “Estimation of Local Strain Fields in Two-Phase Elastic Composite Materials Using UNet-Based Deep Learning,” Integrating Materials and Manufacturing Innovation, vol. 10, pp. 444–460, Sept. 2021.DOI
  12. [12]A. Bhaduri, A. Gupta, and L. Graham-Brady, “Stress field prediction in fiber-reinforced composite materials using a deep learning approach,” Composites Part B: Engineering, vol. 238, p. 109879, June 2022.
  13. [13]A. Gupta, A. Bhaduri, and L. Graham-Brady, “Accelerated multiscale mechanics modeling in a deep learning framework,” Mechanics of Materials, vol. 184, p. 104709, Sept. 2023.DOI
  14. [14]I. Saha, A. Gupta, and L. Graham-Brady, “Prediction of local elasto-plastic stress and strain fields in a two-phase composite microstructure using a deep convolutional neural network,” Computer Methods in Applied Mechanics and Engineering, vol. 421, p. 116816, Mar. 2024.DOI
  15. [15]J. R. Mianoordi, N. H. Siboni, and D. Raabe, “Teaching solid mechanics to artificial intelligence—a fast solver for heterogeneous materials,” npj Computational Materials, vol. 7, p. 99, July 2021.
  16. [16]M. S. Khorrami, J. R. Mianoordi, N. H. Siboni, P. Goyal, B. Svendsen, P. Benner, and D. Raabe, “An artificial neural network for surrogate modeling of stress fields in viscoplastic polycrystalline materials,” npj Computational Materials, vol. 9, p. 37, Mar. 2023.
  17. [17]J. Langcaster, D. Balint, and M. Wenman, “Adapting U-Net for linear elastic stress estimation in polycrystal Zr microstructures,” Mechanics of Materials, vol. 191, p. 104948, Apr. 2024.arxiv.org/abs/2303.08541
  18. [18]Z. Yang, C.-H. Yu, K. Guo, and M. J. Buehler, “End-to-end deep learning method to predict complete strain and stress tensors for complex hierarchical composite microstructures,” Journal of the Mechanics and Physics of Solids, vol. 154, p. 104506, Sept. 2021.DOI
  19. [19]L. Monteiro Fernandes, S. Blusseau, P. Rieder, M. Neumann, V. Schmidt, H. Proudhon, and F. Willot, “A Physics-Informed 3D Surrogate Model for Elastic Fields in Polycrystals,” Computer Methods in Applied Mechanics and Engineering, vol. 441, p. 117944, 2025.DOI
  20. [20]B. P. Croom, M. Berkson, R. K. Mueller, M. Presley, and S. Storck, “Deep learning prediction of stress fields in additively manufactured metals with intricate defect networks,” Mechanics of Materials, vol. 165, p. 104191, Feb. 2022.arxiv.org/abs/2105.10564
  21. [21]Y. Shokrollahi, M. M. Nikahd, K. Gholami, and G. Azamirad, “Deep Learning Techniques for Predicting Stress Fields in Composite Materials: A Superior Alternative to Finite Element Analysis,” Journal of Composites Science, vol. 7, p. 311, July 2023.DOI
  22. [22]Q. Yao, X. Peng, W. Jia, X. Liu, J. Li, and S. Jiang, “Deep Learning Approach for Predicting Stress Fields in Composites With Diverse Fiber Volume Fractions,” Polymer Composites, vol. 46, pp. 14133–14147, Oct. 2025.
  23. [23]Y. Sun, I. Hanhan, M. D. Sangid, and G. Lin, “Learning Hierarchical Material Properties from Microstructure Images in Fiber-Reinforced Polymers Using Convolutional Neural Networks,” Journal of Composites Science, vol. 8, p. 387, Sept. 2024.
  24. [24]R. Sepasdar, A. Karpatne, and M. Shakiba, “A data-driven approach to full-field nonlinear stress distribution and failure pattern prediction in composites using deep learning,” Computer Methods in Applied Mechanics and Engineering, vol. 397, p. 115126, July 2022.DOI
  25. [25]Y. Chen, T. Dodwell, T. Chuaqui, and R. Butler, “Full-field prediction of stress and fracture patterns in composites using deep learning and self-attention,” Engineering Fracture Mechanics, vol. 286, p. 109314, June 2023.
  26. [26]M. Maurizi, C. Gao, and F. Berto, “Predicting stress, strain and deformation fields in materials and structures with graph neural networks,” Scientific Reports, vol. 12, p. 21834, Dec. 2022.DOI
  27. [27]M. Yacouti and M. Shakiba, “Integrated convolutional and graph neural networks for predicting mechanical fields in composite microstructures,” Composites Part A: Applied Science and Manufacturing, vol. 190, p. 108618, Mar. 2025.DOI
  28. [28]M. M. Rashid, T. Pittie, S. Chakraborty, and N. A. Krishnan, “Learning the stress-strain fields in digital composites using Fourier neural operator,” iScience, vol. 25, p. 105452, Nov. 2022.arxiv.org/abs/2207.03239
  29. [29]H. Feng and P. Prabhakar, “Difference-based deep learning framework for stress predictions in heterogeneous media,” Composite Structures, vol. 269, p. 113957, Aug. 2021.DOI
  30. [30]M. Li, Y. Zhang, W. Qu, W. Zhu, and Y. Ke, “A triplet attention-enhanced deep learning approach to predict full-field stress of unidirectional CFRP composites with microvoids,” Composites Science and Technology, vol. 271, p. 111361, Oct. 2025.
  31. [31]X. Wei, L. Zhao, Y. Sun, and X. Zhang, “A deep learning based framework for predicting temperature-dependent mesomechanical fields in particulate composites,” Composite Structures, vol. 378, p. 119930, Feb. 2026.DOI
  32. [32]V. Holla, G. Vu, J. J. Timothy, F. Diewald, C. Gehlen, and G. Meschke, “Computational Generation of Virtual Concrete Mesostructures,” Materials, vol. 14, p. 3782, 2021.DOI
  33. [33]Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, pp. 2278–2324, Nov. 1998.
  34. [34]A. Paszke, S. Gross, F. Massa, A. Lehrer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelsh ein, L. Antiga, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in 33rd Conference on Neural Information Processing Systems, Curran Associates, Inc., 2019.arxiv.org/abs/1912.01703
  35. [35]J. Ansel, E. Yang, H. He, N. Gimelshein, A. Jain, M. Voznesensky, B. Bao, P. Bell, D. Berard, E. Burovski, G. Chauhan, A. Chourdia, W. Constable, A. Desmaison, Z. Devito, E. Ellison, W. Feng, J. Gong, M. Gschwind, B. Hirsh, S. Huang, K. Kalambarkar, L. Kirsch, M. Lazos, M. Lezcano, Y. Liang, J. Liang, Y. Lu, C. K. Luk, B. Maher, Y. Pan, C. Puhrsch, M. Reso, M. Saroufim, M. Y. Siraichi, H. Suk, S. Zhang, M. Suo, P. Tillet, X. Zhao, E. Wang, K. Zhou, R. Zou, X. Wang, A. Mathews, W. Wen, G. Chanan, P. Wu, and S. Chintala, “PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, (La Jolla CA USA), pp. 929–947, ACM, Apr. 2024.DOI
  36. [36]B. Polyak, “Some methods of speeding up the convergence of iteration methods,” USSR Computational Mathematics and Mathematical Physics, vol. 4, pp. 1–17, Jan. 1964.DOI
  37. [37]I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” ICLR 2019, Jan. 2019.
  38. [38]D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” ICLR 2015, Jan. 2017.
  39. [39]K. Jordan, Y. Jin, V. Boza, Y. Jiacheng, F. Cesista, L. Newhouse, and J. Bernstein, “Muon: An optimizer for hidden layers in neural networks,” URL https://kellerjordan.github.io/posts/muon, 2024.
  40. [40]T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, (Anchorage AK USA), pp. 2623–2631, ACM, July 2019.arxiv.org/abs/1907.10902
  41. [41]A. Chen, A. Chow, A. Davidson, A. DCunha, A. Ghodsi, S. A. Hong, A. Konwinski, C. Mewald, S. Murching, T. Nykodym, P. Ogilvie, M. Parkhe, A. Singh, F. Xie, M. Zaharia, R. Zang, J. Zheng, and C. Zumar, “Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle,” in Proceedings of the Fourth International Workshop on Data Management for End-to-End Machine Learning, (Portland OR USA), pp. 1–4, ACM, June 2020.
  42. [42]M. Waskom, “Seaborn: Statistical data visualization,” Journal of Open Source Software, vol. 6, p. 3021, Apr. 2021.DOI
  43. [43]A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. Smola, “A Kernel Two-Sample Test,” Journal of Machine Learning Research, vol. 13, pp. 723–773, 2012.DOI
  44. [44]A. Gupta, I. Saha, L. Graham-Brady, and T. A. Zaki, “Deep learning driven framework for optimization of polycrystalline microstructures under competing strength requirements,” 2026.arxiv.org/abs/2610.09461
  45. [45]J. Liu, J. Su, X. Yao, Z. Jiang, G. Lai, Y. Du, Y. Qin, X. Lu, Y. Lu, Y. Liu, et al., “Muon is scalable for LLM training,” arXiv preprint arXiv:2502.16982, 2025.arxiv.org/abs/2502.16982

Paper details

vlnc:W_stk6azjh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents