WorksPhysics and Astronomy
PINNs-MPF: A Physics-Informed Neural Network Framework for Multi-Phase-Field Simulation of Interface Dynamics
Abstract
We present an application of Physics-Informed Neural Networks to handle MultiPhase-Field simulations of microstructure evolution. It has been showcased that a combination of optimization techniques extended and adapted from the PINNs literature, and the introduction of specific techniques inspired by the MPF Method background, is required. The numerical resolution is realized through a multi-variable time-series problem by using fully discrete resolution. Within each interval, space, time, and phases are treated separately, constituting discrete subdomains. An extended multi-networking concept is implemented to subdivide the simulation domain into multiple batches, with each batch associated with an independent Neural Network trained to predict the solution. To ensure efficient interaction across different phasesand in the spatio-temporal-phasic subdomain, a Master NN handles efficient interaction among the multiple networks, as well as the transfer of learning in different directions. A set of systematic simulations with increasing complexity was performed, that benchmarks various critical aspects of MPF simulations, including different geometries, types of interface dynamics and the evolution of an interfacial triple junction. A comprehensive approach is adopted to specifically focus the attention on the interfacial regions through an automatic and dynamic meshing process, significantly simplifying the tuning of hyper-parameters and serving as a fundamental key for addressing MPF problems using Machine Learning. The pyramidal training approach is proposed to the PINN community as a dual-impact method: it facilitates the initialization of training and allows an extended transfer of learning. The proposed PINNs-MPF framework successfully reproduces benchmark tests with high fidelity and Mean Squared Error loss values ranging from 10 to 10 compared to ground truth solutions.
The PINNs-MPF Framework
Discrete Resolution in Time
Extended domain decomposition
Pyramidal Training and Block Training
Adaptive mesh-free optimization
Adaptive re-meshing
Denoising Loss
Dynamic Resampling
Propagation of BC across Sub-domains
Handling Multi-Phases (Vertical Optimization)
The Wheel of Optimizers
Transfer of Learning
Phase correction
Methods
Multi-Phase-Field Method
Dimensionless form of the PDEs
Results
Simulation Results
Traveling Interface under Constant Driving Forces
Curvature-driven Interface Motion
Establishing Equilibrium Triple Junctions
Discussion
Conclusion
Introduction
Today’s microstructure modeling has grown into an extensive branch of materials research, on the one hand, revealing the multi-physics of the concurrently evolving microstructural constituents across scales and, on the other hand, bridging these with the process, property and performance of materials. Despite remarkable advances over the last two decades, the rapidly growing complexities of materials’ chemistry and processing are constantly surpassing our actual microstructure modeling capacity, thus, decelerating materials innovation. A potential solution for this growing challenge is to utilize the recent developments in machine learning (ML) and deep learning (DL) for microstructure modeling [1]. The recently introduced Physics-Informed Neural Networks (PINNs) are showing promising capabilities in addressing physical phenomena [2]. By enabling non-linear approximations of solutions for Partial Differential Equations (PDEs), the PINNs counterpart of the conventional approaches, such as phase-field, fluid dynamics and finite-element methods, would allow for a mesh-free modeling framework, capable of transfer of the learning and without the necessity of local approximations or simplifications in the underlying physics [2, 3, 4]. These promote new perspectives in microstructure modeling, motivating us to pursue the current line of studies.
First applications of PINNs to solve 1D PDEs (e.g., the heat equation, wave equation and Burgers’ equation) were quite successful, demonstrating its simplicity and efficiency [5, 6, 7]. Since then, deeper applications have been increasingly targeted in various areas including heat transfer [8, 9], solid mechanics [3, 10, 11, 12, 13, 14, 15, 16], electromagnetic [3, 4, 17], turbulence modeling [18, 19], electro-chemistry [20, 21] and Energy [22, 23].
However, as the literature on resolving PDEs using PINNs has grown, several challenges have been revealed, highlighting the necessity for optimization techniques tailored to specific physical contexts. Jagtap et al. [24] proposed Extended Physics-Informed Neural Networks (XPINNs) to address moving boundaries. This method subdivides the domain into different mini-batches to locally compute the solution using distinct neural networks. XPINNs were tested on the one-dimensional viscous Burgers equation, employing various optimization techniques such as subdomain decomposition (in time and space), adaptive activation functions, independent hyper-parameter adjustment for each network, and a specific data processing method to characterize the average behavior of predictions along the subdomain interfaces. Kharazmi et al. [25] introduced hp-Variational Physics-Informed Neural Networks (hp-VPINNs), utilizing domain decomposition and h-refinement with dynamic mesh resolution adjustment to target the advection–diffusion equation (1D) and Poisson equation (1D-2D). Meng et al. [26] propose a Parareal PINNs (PPINN) framework for solving time-dependent PDEs using domain decomposition. The examples include ordinary differential equations (ODEs) and a 2D nonlinear diffusion-reaction equation. They employ various optimization techniques like alternating between Adam and L-BFGS optimizers, Quasi-Monte Carlo sampling for stochastic ODEs, and principally a parallel-in-time training approach with a serial update at the subdomain interfaces using the fine PINN solutions as a parallel prediction-correction process. Penwarden et al. [27] further enhanced these techniques by introducing stacked-decomposition inspired by causality literature (time-causality enforcement methods [28, 29]), window-sweeping, alternance of optimizers and transfer of learning methods proposed to overcome training challenges, respect causality, and improve scalability by limiting computation per optimization iteration. The authors consider the 1D convection equation, 1D Allen-Cahn equation and 1D Korteweg–de Vries. While the reported methods have demonstrated promising results, further testing and extension are necessary to address more complex scenarios, such as two-dimensional configurations involving systems of equations, Multi-Phase Field problems and three-dimensional domains. These scenarios introduce additional challenges that require innovative approaches and optimization techniques.
A few applications of PINNs to handle diffuse interfaces were recently presented, especially to study evolution of multiple interacting phases or components [30, 31, 32, 33, 34, 35]. Zheng et al. [36] have developed a physics-constrained neural network (PCNN) to predict the sequential motions of flow simulations. Due to the challenging nature of multi-phase problems, the applied approach was based on the satisfaction of the mass conversation constraint when encoding the observation of the recurrent NN. A conservative boundedness mapping algorithm (MCBOM) was then called to correct the phase prediction, reported to be efficient for handling sequential notions in some testing benchmarks in 2D, e.g., the evolution of three phases in the shear layer and dam break problems. Haghighat et al. [32] proposed a dimensionless form of the PDEs combined with a sequential training strategy based on stress-split algorithms and multi-networking, applied for solving several problems in poroelasticity. It was found that both the size and structure of the NN, along with the hyperparameters of the optimizer, such as the learning rate, can significantly influence the quality of PINN solutions. Amini et al. [34] introduced inverse modeling of nonisothermal multiphase poromechanics using PINNs, successfully applying the method to both single- and multi-phase flow in porous media.
During these remarkable advances, several challenges were also spotted in the field that remain to be addressed. A primary concern with PINNs involves the gradient pathology in the total loss function, due to larger gradients in higher-derivative terms, affecting the accuracy of the predictions [32]. Furthermore, ensuring systematic convergence of training becomes difficult, especially with the high computational cost of multi-networking. The use of a multi-networking approach can generate differing architectures or training data among separate NNs, which leads to imperfect alignment and thus discontinuities or artifacts along boundaries. Adjusting the size of training sets, including initial conditions, collocation points, and boundary conditions, was shown to help resolve this problem, but at the same time, this adds to the complexity and computational costs [2, 37]. The assembly and transfer of collective learning from multiple networks (transferring the learning) is another major challenge in PINNs. Here tuning hyper-parameters, even with a single NN, remains challenging: Adaptive weighting, while introduced to address feature dependencies, reflects non-systematic learning of solution patterns [20]. In addition to these, the profound sensitivity of the solution to the hyper-parameters, encompassing variables such as the number of neurons, network layers, learning rate, and the composition of training datasets, emerges as a major issue that requires more attention when dealing with PINNs [20, 38]. Regarding Multi-phase Field (MPF) equations, they involve a coupled system of nonlinear PDEs with multiple phases and several parameters. The presence of higher-order derivative terms and a non-convex potential term further contributes to additional non-linearities and the emergence of several local minima. Additionally, the generated solutions often require correction algorithms to satisfy conservation laws, rendering classical PINNs unable to make accurate predictions. Therefore, incorporating conservative correction algorithms and adequate optimization methods becomes necessary [39].
Multi-phase-field methods (MPFM) for microstructure modeling have proven to be powerful tools for capturing intricate multi-physics phenomena in multi-phase, multi-component materials [40, 41, 42]. Being based on a generalizable free energy functional, a key feature of the MPF framework is its capacity to be flexibly expanded to different degrees of complexity which can be specific to the physics of the problem at hand. A representative demonstration of this flexibility has been explored in dealing with polycrystalline microstructures: Starting from curvature-driven interface kinetics, the MPF framework has gradually expanded to cover the chemical [43], mechanical [44] and even the physics of magnetic and electric contributions [45]. These led to the accurate investigation of complex phenomena such as precipitation, recrystallization, and phase transformation as well as various chemomechanically coupled phenomena [46, 47, 48]. Despite successful implementations and applications [49, 50], an intrinsic challenge of the MPF approach is that with any further advancement and raising scientific questions, immediate needs for extensive programming and optimization efforts are required, which often spread to deep levels of redefining key parameters and functions as well as memory- and storage-structures in the code. These originate from the fact that any update to the model necessitates changes to the underlying free energy functional and corresponding equations of motion. As a result, it becomes difficult to keep the software updates up with the pace of emerging topics in materials research, for instance, the unanswered questions regarding non-equilibrium microstructure evolution in additively manufactured materials [51, 52] or the complex diffusion-interface couplings in novel high-entropy alloys [53, 54].
This work presents a comprehensive framework, termed PINNs-MPF, specifically designed to tackle the challenges of solving MPF equations using PINNs. By identifying the limitations of existing PINN methods when applied to MPF equations and recognizing the additional complexities introduced by these non-linear, convex, and constrained systems, we propose a unified approach that combines and extends relevant PINN techniques with MPF method strategies. This strategy is inspired by the core development of the OpenPhase software for MPF simulations, which has demonstrated success in studying diffuse interface problems, but aiming to overcome reported limitations, by enabling a learnable solution while efficiently handling non-linearities, to integrate more physics. [46]
This framework introduces key novelties through its optimization strategies: (i) A extended domain decomposition approach is implemented that uses a central coordinating network (referred to as the "Master") to manage tasks and ensure continuity among subnetworks (referred to as the "workers") during full parallel training in space, time and phases. (ii) An extreme mesh refinement approach, with a distinct focus on the phase-field interfacial zones, is applied. (iii) We simplify and reduce the hyper-parameters as elaborated in the following. (iv) A synchronization step to ensure efficient interactions between the evolving phase-fields is applied. Finally, (v) a pyramidal training approach is applied as a robust alternative to adaptive weighting and an efficient solution for transferring learning of multiple-to-multiple and single-to-multiple networks. Inspired by the MPF benchmark philosophy [46], we test our PINNs across four core applications that built the MPF concept with a progressive complexity: First, we model the motion of a diffuse interface (propagation of a stable phase-field profile), second and third, we study the curvature-driven shrinkage of a grain in the presence and absence of a driving force, and fourth, we investigate the evolution of a triple junction that demonstrates the core principles of a multi-phase evolution. The results are compared against the outputs from OpenPhase and discussed in the context of our hyper-parameters and concepts implemented within our PINNs framework.
The PINNs-MPF Framework
The MPF temporal evolution is governed by the set of PDEs as follows:
with phase-fields , the interface mobility, the interface energy and the interface width. The pairwise term counts for any additional driving force (e.g. chemical, elastic, etc.) influencing the interface dynamics. At every point in space and time, the summation of phasefields/order parameter reads:
with the number of phase-fields present. Further details of the MPF model are given in the Methods § 8. The PINNs-MPF framework is targeted to resolve Eqs. (1) and (2) in time and space and for various initializations. In doing so, PINNs aim to minimize a loss function with respect to the hyper-parameters . With the MPF model in hand, this is composed of several terms:
where is the loss associated with enforcing the governing physics through PDEs:
in which, represents the residual of the PDEs or the physical loss (Eq. (1)) and the number of collocation points. is the loss associated with enforcing boundary conditions (BC) on the simulation domain:
with representing the approximated solution at boundaries and time using the neural network with parameters and, is the prescribed or observed value at the corresponding boundary and time. is the loss associated with enforcing initial conditions (IC), i.e., the spatial configuration of the phase-fields:
that compares the approximated solution at initial state and time against the prescribed or observed value at the corresponding (initial) state . Finally, is the loss associated with the constraint Eq. (2) of the phase summation:
where is the total number of phases in the given point. Unlike the other losses which are computed over a subpart of the domain, the loss over the sum constraint is computed for the entire simulation domain.
The coefficients , , and in Eq. (3) are weight factors associated with different loss terms, to balance the importance of each term. The objective of Eq. (3) is to find the optimal parameters that minimize the whole loss function, resulting in learned neural networks that are then capable of approximating the MPF solutions of a problem with the given initialization and boundary conditions. We note that since the MPF method is based on a free energy functional, reinforcing terms to account for the energy dissipation [55] could be directly added to the loss function, further elaborated in the Discussion (§ 10).
The architecture of the current PINNs-MPF framework is modular and composed of various implementations. To better illustrate the PINNs-MPF framework, Figure 1 presents a flowchart of the framework.
Figure 1. (a) Global architecture of the PINNs-MPF framework. A general description of the discrete training subdomain is provided in (), while a detailed illustration of the training process within each time interval is shown in ().
(b) Pyramidal training and continuity across domains: A comprehensive description of the pyramidal transfer of learning from one subdivision level to another is depicted in (), while an illustrative example of this hierarchical approach, followed by addressing continuity between domains, is provided in ()
Figure 1: The architecture of PINNs-MPF framework. Panel (b) adds some details to panel (a). The pyramidal training enables continuity and multiple-to-multiple transfer of learning. The continuity of the prediction is ensured by the propagation of boundary predictions. The set of techniques employed in this model is accessible within the code. The user has the flexibility to reserve the entire or a specific subset of the options, based on the simulation requirements and complexities.
Panel (a) in Figure 1 presents the global architecture: Similar to the conventional MPF method, full discrete training in space and time is considered. Our experiments demonstrate that this approach not only eases the comparison to our reference MPF solutions but also allows for maintaining a constant demand for resources throughout the entire simulation, in contrast to continuous training, which demands growing consumption [2]. The general control of the training is handled through a master network, referred to as the MASTER-PINN, while the workload is shared among multiple parallel NNs (subdomains), referred to as WORKER PINNs (c.f. paragraph 2.2). The subdividing procedure is automated. Within each subdomain, the training methodology is carried out independently, employing all distinctive techniques as introduced in the following. This includes ‘pyramidal training’, a structured approach designed for handling the transfer of learning along the spatial domain (c.f. paragraph 2.3). The term ‘basket of PINNs’ (c.f. paragraph 2.7) refers to an ensemble of networks dynamically selected for optimization to enhance efficiency. Indeed, Without an efficient strategy, the resource demand can grow exponentially, leading to impractical training times and excessive use of computational resources. Additionally, the concept of the ’wheel of optimizers’ (c.f. paragraph 2.7) is introduced, denoting the interaction between the optimizers.
The training is in two directions: First in space, where the domain is decomposed into regular boxes (denoted as ). The degree of this decomposition depends on the complexity of the problem. The second direction of training concerns the phase-fields. For each of these spatial-temporal blocks, referred to as a batch, a single WORKER PINN, referred to as , is trained to locally predict the solution. Hereafter, the term "blocks" is employed interchangeably with "quarters," as the spatial domain is consistently divided into four blocks. Each handles a given phase in a given batch and, depending on the domain decomposition, interacts along boundaries for spatial continuity.
At the beginning of each training (time) step, the batch structure is separately built for each phase-field , i.e., PINNs are created and inherit the same architecture as the MASTER-PINN. During the training, several optimization techniques could be applied (based on the initial user selection), including pyramidal training, the wheel of optimizers, the basket of s, and the concept of Quarters-based training. An upper view of the global algorithm is shown in Algorithm 1, where , denote the minimum number of batches (coarser subdivision in case of pyramidal training) and the periodic epoch for scipy optimization respectively. In the following, we describe the details of various implementations of the PINNs-MPF framework and the related algorithm.
Discrete Resolution in Time
We consider the MPF equations as solving a multi-variable time-series problem through the NNs. This has also been discussed in previous studies [56, 57, 58, 59]. The computational domain in the upper left corner panel in Figure 1(a) depicts this idea. Here the training process occurs in spatial domains/subdomains at fixed time intervals, analogous to time steps in conventional modeling methods, but with relatively larger intervals to harness the robust linearization capabilities of deep learning (DL). In order to proceed to a subsequent time domain, a global loss threshold is predefined to be achieved. This discrete resolution ensures a quasi-constant resource consumption, hence preventing any accumulating computational load along the time axis. To draw an analogy between discrete resolution in MPF modeling, the notation (also denoted for dimensionless resolution) is adopted here for both PINNs and MPF resolutions; in the context of PINNs, this notation refers to the length of the time interval. In the context of the PINN literature, this approach categorizes our framework within the
Input1: Initialization
Input2: USER inputs
1.1 Initialize the Master PINN $\leftarrow$ User inputs
1.2 Preapre Global training data $\leftarrow$ Initialization + USER inputs
//
1.3 for epochs in range $N_{\mathrm{epochs}}$ (total number of epochs) do
1.4 set $t_{\min}$ and $t_{\max}$
// $\to$ training subdomain bounds
1.5 while $N_{\mathrm{batches}} \succeq N_{\mathrm{batches}\ \min}$ do
1.6 initialize the multi-networks/pinns.
1.7 decompose the subdomain.
1.8 $\to$ Horizontal subdivision (initialization of $PINN_i$s handling first phase).
1.9 + Vertical extrusion (initialization of $PINN_i$s handling other phases).
1.10 affect IC training data to each $PINN_i$).
1.11 $\to$ dynamically generate BC and collocation points for each $PINN_i$.
1.12 divide the subdomain into different training Blocks/Quarters.
1.13 assign each $PINN_i$ to a parent Block.
1.14 for each Block do
1.15 for each $PINN_i$ in Block do
// Remind: each $PINN_i$ handles one different phase
1.16 if first phase then
1.17 select a candidate $PINN_i \leftarrow$ highest ratio of interfacial points.
1.18 $\to$ put "selected $PINN_i$" in the Basket.
1.19 store information about "selected $PINN_i$" (dictionary). $\to$ to use if
pyramidal training
1.20 else
1.21 for each phase in current Block do
1.22 get $\leftarrow$ the NN with the same batch index as the "selected $PINN_i$".
// $\Rightarrow$ Basket of $PINN_i$s Ready
1.23 Multi-processing $\to$ start Parallel Training $\to$ parallel optimization using Adam.
1.24 if epoch % $scipy_{\mathrm{epoch}}$ then
1.25 if $pinn.loss \succeq Threshold$ for $pinn$ in $pinns$ then
1.26 apply L-BFGS-B optimization.
1.27 resample collocation points.
1.28 if $PINN_i.loss \prec Threshold$ for $PINN_i$ in $PINN_i$s then
1.29 if $N_{\mathrm{batches}} \succ N_{\mathrm{batches}\ \min}$ then
1.30 reduce $N_{\mathrm{batches}} \leftarrow$ Eq. 8
// $\to$ Pyramidal training.
1.31 increase $t_{\min}$ and $t_{\max}$
// $\to$ go to the next training subdomain
Figure 1 (PDF p. 9). Algorithm 1: General training scheme for the implemented PINNs-MPF.
class of time-marching or temporal PINNs, as discussed in the causality literature [27, 60, 61].
Extended domain decomposition
This technique extends the domain decomposition approach introduced in previous PINN studies [24, 26, 62] by adding a third dimension of decomposition and parallelization along the phases direction. This extension aims to handle the increased complexity of the target problem more effectively. A centralized network ensures synchronization among the multi-networks, facilitating a coordinated learning process across the decomposed spatial, temporal and phase domains. Each phase, managed by a corresponding Neural Network, requires concurrent access to the interaction terms () of other phases (Eq. (15)), which collectively form dual or higher-order junctions. These interaction terms demand a synchronization method (Algorithm 2). Consequently, parallel implementation is crucial for the precise calculation of interaction terms, as per the NN definitions. A serial approach cannot adequately manage these interactions, rendering parallelism indispensable rather than merely an improvement to be quantitatively compared against serial execution or existing methods, if available.
One challenging aspect of a PINNs-MPF is the heavy computational costs associated with the size of the simulation domain and the number of phase-fields. Here the concept of multi-networking is essential to distribute this computational load efficiently. This involves the parallelization of multiple s and their associated subdomains obtained through a structured domain decomposition. In this context, a MASTER-PINN is designed to distribute tasks, training data, and collate the learned outcomes from the WORKER-PINNs/s. Each operates independently and concurrently, utilizing a dedicated thread in the computer. It functions indeed as an autonomous entity responsible for processing a specific batch within the spatial domain and managing a given phase-field within the MPF domain. All s are assumed to have the same architecture (number of layers and neurons per layer, learning rate, etc.). This facilitates complete parallelization of the training process and ensures spatial continuity throughout the training.
As depicted in the flowchart in Figure 1, the MASTER-PINN does not undergo any training but performs several central tasks, including (i) preparing the training data set, (ii) initializing and assigning each WORKER for training, (iii) initiating and controlling the training process, (iv) facilitating communication among the WORKER PINNs, (v) managing the temporal transition of the training, and (vi) transferring the (ideally) successful learning from the actual training to another simulation. From a technical point of view, it is noteworthy that the current multi-networking approach enables the transfer of learning from multiple networks to a single network. This capability is available within the code through the pyramidal training option, wherein the subdivision could be progressively transformed from fine to coarse, ideally converging to a single network (c.f. paragraph 2.3).
The communication between s could be classified into two categories: a spatial ‘horizontal’ communication which is required to ensure spatial continuity across the boundary between their subdomains (c.f paragraph 2.4) and a phase-field ‘vertical’ communication that allows the coupled co-evolution of the phase-fields in the context of the MPF model (c.f paragraph 2.6). To facilitate the training of the WORKER-PINNs and their efficient management by the MASTER-PINN, the concept of the trinity batch--phase is crucial. Each is assigned a unique identifier by associating it with a specific batch number and phase-field number, as illustrated in Figure 1a. Consequently, the local neural network incorporates horizontally stacked spatial-temporal coordinates (), the output (prediction) of which is a single array representing the fraction of the corresponding phase, ranging from 0 to 1. By simplifying the multi-phase problem into discrete single-phase problems in this manner, the computational approach is streamlined. This necessitates efficient interactions between the phases, which is treated through the ‘vertical’ communication. Finally, it is preferred to spatially decompose the simulation into equal batches.
Pyramidal Training and Block Training
Merging the hyper-parameters of various s (WORKERS) at once and with equal significance can result in overwriting certain features in the domain. For this purpose, we have thought of a gradual pyramidal training and merging. The concept of pyramidal training draws inspiration from the data augmentation principle in ML [63, 64]. It is based on the assumption that adding more data to a pre-trained model serves as a warm starting point, enabling the NN to better capture crucial features before further data processing. We propose to implement a pyramidal training paradigm in conjunction with another concept of ‘training on blocks’. These two complementary concepts are showcased in Figure 1(b) and described as follows: Once the spatial domain is subdivided into multiple batches, each one can be further divided into distinct domains, giving a pyramidal structure. Each of these finer domains in the pyramidal structure is called a block or a quarter. The procedure of subdivision can continue as depicted in 1(b). After the pyramidal subdivision is completed, the training process initiates at the finest subdivision level (i.e., the base of the pyramid). However, instead of training all Neural Networks (NNs) within all batches, within each block/quarter, with the highest number of interfacial points is selected for training; this selection concerns the first phase and the selected s serve as a reference for other phases. Similarly, for the other phases (in the vertical direction), training is assigned to the within the same batch as the reference within each block/quarter. This selective handling reduces the computational costs and makes the training focused on hot spots in the simulation domain. Once target losses are reached for the selected s, the training stops at the finer level and transitions to a coarser level using the same algorithm described above. The training on the coarser level begins with a warm-start approach, where each selected receives learning from the previously selected at the finer level, within the same quarter and sharing a spatial intersection area, thereby achieving a data augmentation-like training. The transition from one level of the pyramid to another is set by a squared relationship to ensure a pyramid-like subdivision:
where is the number of batches in the next upper level until ( is reached, that means the continuation of training the s on the whole spatial domain. This gradual resolution has two main advantages: On the one hand, it allows a fast identification of the optimal set of hyper-parameters, and therefore key features, without the need for an adaptive weighting strategy. On the other hand, it guarantees the training of the same numbers of NN independently of the number of subdivisions (). For that, it is worth mentioning that such a strategy is highly recommended to be activated for the early stages of the training and then set off to speed up the training.
Adaptive mesh-free optimization
The Adaptive mesh-free optimization (AMFO) is here introduced as a PINN implementation extending the adaptive mesh refinement (AMR) concept, from the phase-field literature, to dynamically adjust collocation point distributions. AMR variants have indeed proven valuable for phase-field and MPF models, allowing efficient capture of evolving interfaces and steep gradients. By incorporating adaptive collocation point sampling, this approach leverages the mesh-free nature of PINNs while retaining AMR benefits like improved accuracy and robustness [65, 66, 67, 68, 69]. The ease of integrating adaptive sampling with PINNs makes this approach attractive for tackling challenges posed by phase-field models, such as complex geometries and localized high-gradient regions. The proposed technique employs adaptive collocation point sampling to concentrate points around interfacial regions, a denoising loss function to reduce noise in non-interfacial areas, and dynamic resampling to continuously adapt the overall collocation point distribution.
Adaptive re-meshing
In the presence of diffuse interfaces, continuity among the phase-fields and across the spatial domains is required. Inspired by the storage concept proposed for the OpenPhase [46], we adopt a mesh structure that mainly focuses on the interfacial regions, neglecting interior grain zones where phase-fields are equal to 0 or 1. This strategy not only reduces the computational costs but also reduces the tendency for an error when coping for the continuity across the domains/batches. This is further promoted by the high capacity of fitting and extrapolation of ML which we elaborate on later in the Discussion (§ 10). To this end, some key hyper-parameters are user-predefined as follows. First, the minimum and maximum number of IC points per batch and are defined. Once the ratios of 0 and 1 phase-values per batch are given, the number of IC points per batch is set, following:
in which donates the percentage of interfacial points per batch. The sigmoid-like function in Eq. (9) allows gradual densification of the mesh depending on the importance of the zone, so that the diffuse interfacial area and triple junctions in a multi-phase configuration should be allocated by the higher number of IC points. Using Eq. (9), the collocation points are generated dynamically around the IC points (typically within a circle of radius ) through a predefined ratio to get , where is the number of collocation points per batch and is a user parameter that could be adjusted depending on the nature of the problem. Experimenting with our benchmark studies below, we varied this coefficient between 20 to 40. Another potential improvement, to reduce the hyper-parameters in Eq. (9) and to increase the significance to triple junctions, is to define the density of interfacial points dynamically. This would involve computing the percentage of interfacial points per batch dynamically, thereby determining the number of interfacial points per batch as follows:
where represent the limits of each . Recall that corresponds to the associated percentage of interfacial points, emphasizing that denotes the interfacial width, and these parameters are already available within the code. Additionally, is an introduced hyper-parameter (user input) representing the local density of the mesh, indicating how much the mesh should be densified based on the number of interfacial points inside. With this improvement, the geometric hyper-parameters are reduced to two variables: the number of subdivisions () and the local density (), considering that the coefficient could be set at 20 for all numerical experiments. This simplification allows focusing on the architecture of the neural networks. The s sharing the same batch index while handling different phases should use common collocation data (common batches). This introduces an additional stage of optimization. Further details on this can be found in the Supplementary Material (SM), section C.
Denoising Loss
Focusing on the diffuse interfacial zones gives an efficient and precise approximation of the MPF solution, but still does not exclude random prediction away from the interfaces. To address this issue, a denoising loss, functioning as a corrector, is introduced. A batch is typically made of three distinct regions: A, B, and C, where region A corresponds to the interface, region B is the grain-containing area, and region C encompasses areas far from the interface (no-grain), see Figure 2. By dynamically identifying areas of interest, as depicted in Figure 2, the additional loss term is applied within unlabeled regions that are free of collocation points, such that, each operates to minimize the prediction of phase-field values in regions identified as ’no grain’ while maximizing predictions in the grain-containing regions. Eq. (11) describes this loss, where the Mean Squared Error (MSE) is calculated based on the prediction and is conditioned on whether the region is identified as ’no grain’ or ’grain,’ denoted by and , respectively:

Figure 2. Illustration of the denoising loss which is computed in the identified areas B and C, far from the interface.
Dynamic Resampling
Resampling, applied after each Scipy optimization, serves as a dynamic process that benefits both converged and non-converged s. The collocation points undergo automatic resampling. This approach offers a dual purpose: converged s are subjected to testing with new but relatively similar populations. This testing mechanism safeguards against overfitting and ensures the robustness of the predictions as confirmed by previous PINN studies [5, 70, 71, 72, 73]. Simultaneously, the nonconverged s are exposed to new, unexplored samples to have a better chance to enrich their training and therefore a better convergence.
Propagation of BC across Sub-domains
As shown in Figure 1(b), the propagation of BC along the neighbors in our decomposed space is required to ensure continuity and therefore a reliable prediction of the MPF solution. To this end, a customized BC loss is introduced that concerns the spatial internal boundaries across sub-domains/batches/s handling a given phase-field. Here, each optimizes its weights by identifying its neighbors while considering the boundary predictions of its neighbors that are used to minimize the additional BC loss. The BC loss function is set as the sum of differences between the prediction of the current and those of the neighbors, depicted also in Figure 1b. A detail that enhances the handling of the boundary condition is to consider the (in batch 0 for each phase) as an absolute reference after addressing its PDE and IC losses. Subsequently, for a given , the west and inner neighbors are regarded as temporal references. This leads to a systematic propagation of the boundary conditions, ensuring that all instances update their weights based on the . These interactions create mutual dependencies among neighboring domains/batches/s, fostering spatial-temporal continuity during the training process.
Handling Multi-Phases (Vertical Optimization)
The full discrete construction of the current framework in space, time and phase-fields allows the instant communication between s, through the , that is required to perform the multi-phase studies. This is achieved in two steps: In the first step, one is entailed to identify the possible interaction(s) between any two phase-fields. This must be done for every . Obviously, with every interaction found, an associated for that phase-field is identified as well. A fundamental assumption here is that two given phase-fields interact when their diffuse interfaces intersect. This results in obtaining a batch-phase space. In 2D, a maximum of three phase-fields intersect, forming a triple junction. An example of this idea is illustrated in Figure 3 where we find that out of the 48 possible batch-phase combinations, 8 exhibited no interactions. This approach has a significant impact when handling a large number of phase-fields. Hereafter, the term ’grain’ is also used interchangeably with ’phase’ once reflects the same entity.
Figure 3. Illustration of the interaction between different phases in different batches for a 2D triple junction case: each phase, in each batch interacts with up to three phases.
The - structure enables a computational- and memory-efficient calculation of the Laplacian term in Eq. (1). Each responsible for a specific phase-field communicates its computed Laplacian term instantly to the neighboring s working on the same batch but handling different phase-fields. This is essential for avoiding the computation of multiple Laplacian terms by each and enhancing the computational efficiency. The numerical implementation of the MPF model (Eq. (1)) is simplified into the parallel computation of the interaction terms for each phase within each batch. Subsequently, a robust communication protocol ensures synchronization of the computation for the correct equation terms, as detailed in Algorithm 2. To ensure synchronization, a given needs to wait to receive all the required information from the s in interactions. As a reminder (c.f. paragraph 2.4), an optimization detail involves the use of a shared collocation dataset for all s handling different phases within the same batch. For simplification, the collocation dataset corresponding to the first phase in the initialization (phase in algorithms 1 and 2) is consistently selected as a reference.
Input1: PDE equation (1) Input2: Interaction infos (dictionary) 2.1 if then 2.2 get self 2.3 else 2.4 identify the other in interaction with self. 2.5 get of the handling the phase 1. 2.6 compute 2.7 compute self interaction term () (15) 2.8 for in s in interaction do // 2.9 while is not updated do 2.10 wait 2.11 for in s in interaction do // 2.12 while is not updated do 2.13 wait 2.14 All interaction terms are computed 2.15 compute the right side of (1) 2.16 compute the PDE loss for the actual batch (4) 2.17 correct the phase predictions ; 2.18 compute the sum constraint loss for the entire grid at (7) 2.19 return the total loss
Algorithm 2: The vertical optimization along phases to optimize the computing of the PDE loss for a given s (prediction a given phase () in a given batch). Here "self" denotes the considered while represents the associated dataset of collocation points. The dynamic correction of the phase summations is done within this block.
The Wheel of Optimizers
The framework effectively combines Adam’s stochastic optimization with Scipy’s L-BFGS, fostering a collaborative approach. Adam operates at each epoch, contributing its stochastic optimization capabilities, while Scipy, notably the L-BFGS optimizer, is periodically selected. During these selections, L-BFGS efficiently leverages information from the Hessian matrix to optimize weights. When a given successfully converges, reaching a loss value below the predetermined threshold, it temporarily steps back, entering a wait state. Meanwhile, other s, selected from a shared ’basket of s,’ proceed with their L-BFGS optimization. This metaphorical basket illustrates the practice of choosing specific PINNs for weight optimization and then reintegrating them, symbolizing a cyclic process; hence, the terminology ’wheel of optimizers’. The purpose of this cyclic process is to ensure that optimized learning results in synchronized motion between batches, leading to accurate predictions at boundaries, and effectively captures the dynamics across phases, including terms from PDE.
Transfer of Learning
The highly symmetric and parallel construction of the current implementation enables the transfer of learning in multiple directions. Within the same simulation, we transfer the learning spatially through the pyramidal training (between the levels of pyramids) and across boundary conditions. The transfer also occurs temporally, from one spatial domain to its image in the next time step. These allow us to create checkpoints to continue/restart a simulation without losing the accumulated learning. A particular application of such transfer is to optimize the parallel performance of the worker s. When a worker fails to converge after a certain number of Scipy optimizations, it is then warm-started by receiving the learning from the nearest neighbor already converged. This is to enhance its convergence rate and overall performance. Such a situation could arise indeed due to various factors, including insufficient data or training samples, numerical instabilities, singularities or discontinuities in the data and inadequate regularization. Additionally, the framework allows real-time processing, facilitating easy debugging and dynamic adaptability.
Phase correction
As a requirement to handle MPF problems, a dynamic correction mechanism for phase predictions is hereafter introduced. This algorithm conserves the summation of the phase fields of different phases within interfacial regions. PINNs often fail to predict correct solutions, necessitating correction algorithms to satisfy conservation laws [39]. Therefore, incorporating conservative correction algorithms and adequate optimization methods is essential for accurate predictions [36].
Methods
Multi-Phase-Field Method
The free energy functional for a polycrystalline domain is written as
with is the grain boundary free energy density and the is to emphasize the presence of other energetic contributions (e.g. chemical and elastic energy densities) which are not specifically considered here. The grain boundary free energy density that is given as [42]
Here is the non-conservative phase-field variable corresponding to the phase (grain) (idem for ), is the grain boundary energy and is the width of the grain boundary interface . The product accounts for the diffuse nature of the grain boundary and its sum asymptotically grows when going from a simple grain boundary to three and higher order junctions.
The evolution of the different grains is determined by the following system of equations:
with defined as pairwise grain boundary mobilities. Inserting the free energy functional , one obtains
Here the additional term accounts for the non-interfacial contributions to the free energy functional, referred to as a driving force throughout this paper. Further, we have
The phase-field variables are assumed to respect the sum constraint:
which is included in the training as an additional but critical loss term. Indeed, the phase prediction should be dynamically divided by the sum of the order parameters of all phases.
In our simulations, we have considered isotropic interfacial mobility () and energy () also a constant driving force. This results to the simplified form of , presented in Eq. (1). The latter consideration simplifies where is a constant value. In case of a single grain boundary (two phase-fields system):
with
and the kinetic coefficient of the interface :
For a triple junction, the Young’s law becomes
Dimensionless form of the PDEs
The convergence and training speeds of the model are strongly linked to the chosen physical parameters. The use of dimensionless representations for the PDEs is recommended when dealing with slow training. The non-dimensional time, spatial coordinate, temporal derivative, and dimensionless gradient as proposed as follows.
where the time characteristic is identified:
while represents the characteristic length. Thus, the MPF model, as represented by Eq. (1), can be simplified for numerical purposes:
with:
Similarly, the dual-phase case (Eq. (17)) could be simplified to:
Results
Simulation Results
Analogous to the MPF approach, the reliability and performance of a PINNs-MPF for studying the interface dynamics in polycrystalline materials need to be benchmarked capturing three basic aspects: First, the curvature-driven motion of an interface, second, the correct motion of the interface under any additional driving force and third, the force balance at the junctions of interfaces [46]. The first two requirements in this set can be generally written as:
with the local interface velocity normal to its plane, the local curvature of the interface, and a constant driving force, where with consider in Eq. (1). The third requirement concerning interface junctions necessitates the MPF to fulfill Young’s law, which relates the angles between interfaces at the junction to the interfacial energy. For a triple junction and isotropic interface energy assumed, the three interfaces at equilibrium meet at a angle. To test the PINNs-MPF framework, certain simulation scenarios related to the above requirements are studied as listed in Table 1, along with related physical parameters and applied hyper-parameters in the PINNs implementation in Table 2. We note that for the grain shrink scenarios, the number of IC points per batch per batch serves as an input parameter, and Equation (9) is used for its generation (with min and max varied up to 90). However, for the triple junction scenario, this process is automated through Equation (10), where is set to 40 in order to dynamically populate the batches during training, depending on the evolution of the solution.
| Benchmark | dx | dy | Interface width | Mobility | Interfacial energy | Phases | |||
| Traveling wave interface | 100 | 0.01 | 100 | 0.01 | 1000 | 10dx | 1e-4 | 1 | 1 |
| Grain shrinkage under driving force | 64 | .. | 64 | .. | 100 | 7dx | 1e-4 | 1 | 1 |
| Curvature-driven grain shrinkage | 64 | .. | 64 | .. | 150 | 4-9dx | 1e-4 | 1 | 1 |
| Triple junction | 64 | .. | 64 | .. | up to 200 | 6dx | 1e-4 | 1 | 4 |
Table 1. Numerical and physical parameters of the testing benchmarks. The corresponding units of the interface width, mobility, and interfacial energies are respectively m, , and .
| Benchmark | NNs | HL/NN | Nodes / layer | LR | Activation function | max Iter | Per. |
| Traveling wave interface | 1 | 6 | 32 | 1e-3 | tanh + sigmoid (last layer) | 1000 | 100 |
| Grain shrink under driving force | 4 | 6 | 32-128 | 1e-4 | idem | 1000 | 50 |
| Grain shrink with natura motion | 4 | 6 | 128 | 1e-4 | idem | 1000 | 50 |
| Triple junction (*) | 16 (**) | 6 | 128 | 1e-5 | idem | 500 | 50 |
Table 2. Hyper-parameters of the implemented PINN for the different testing benchmarks. LR: Learning Rate, Per.: Periodicity/Epochs
Table 2: Hyper-parameters of the implemented PINN for the different testing benchmarks. LR: Learning Rate, Per.: Periodicity/Epochs
(*) For the triple junction, the pyramidal approach was tested for the early stages of the training. (**) When using the pyramidal training, 64 NNs are inititalized, while 16 NNs are selected for training using the the concept of the basket of PINNs.
To further highlight the effectiveness of multi-networking in conjunction with a combination of optimizers for training PINNs, an additional numerical experiment is proposed as follows: reproducing the benchmark of grain shrinkage under constant driving forces with varying numbers of hidden layers and neurons per layer. Specifically, a fixed number of six hidden layers with the number of neurons ranging from 64 to 512 was applied. This was achieved by employing a cyclic optimization strategy alternating between the Adam optimizer and the L-BFGS optimizer (referred to as the wheel of optimizers). Associated findings and comments are provided in the discussion section. Additionally, we illustrate in Table 3 the progressive increase of the optimization techniques to deal with the induced complexity by the tackled scenarios.
| Benchmark | Activated Techniques | Number of optimization Techniques | Number of trainable parameters |
| Travelling wave interface | Wheel of optimizers (Adam and LFBGS), resampling, discrete or continuous resolution (optional), transfer of learning | 4 | 3,297 |
| Grain shrinkage under driving force | Wheel of optimizers, discrete resolution, extended domain decomposition, Adaptive Mesh-Free Optimization (AMFO), transfer of learning | 5 | 416,005 |
| Curvature-driven grain shrinkage | Wheel of optimizers, discrete resolution, extended domain decomposition, AMFO, transfer of learning | 5 | 416,005 |
| Triple junction | Wheel of optimizers, discrete resolution, Extended domain decomposition, transfer of learning, AMFO, Basket of PINNs, Handling multi-phases, pyramidal training correction of phases predictions | 9 | 1,414,417 |
Table 3. Enumeration of applied optimization techniques with increasing complexity of target problems, as well as the associated number of model trainable parameters.
Table 3: Enumeration of applied optimization techniques with increasing complexity of target problems, as well as the associated number of model trainable parameters.
Furthermore, for the scenarios involving grain shrinking and triple-junctions, the MPF solutions from OpenPhase calculations are used [46]. The associated scheme is provided in the SM, section A. All numerical implementations were coded using TensorFlow and performed on standard workstations with an AMD Ryzen Threadripper PRO 5975WX 32-Cores CPU (64 threads).
Traveling Interface under Constant Driving Forces
As a key starting point, the behavior of a planar interface traveling under a given driving force is explored. In the absence of the curvature, this can be simplified to a 1D problem in which the phase-field profile propagates according to [42, 46]:
The results of this campaign are gathered in Figure 4. The utility of analyzing a traveling wave solution lies in its ability to accurately predict the shape and velocity of the phase-field profile. Figure 4(a) presents the setup of the BC and IC points, as well as the placement of PDE collocation points for this problem. As described in paragraph 2.4, the meshing is predominantly focused within the interface regions, where the PINN model is trained to predict the solution. Excellent agreement is obtained between the PINNs-MPF and the theoretical solutions at various time instants, Figure 4(b). Figure 4(c) shows the spatial evolution of the interface resolved by the PINNs-MPF. To trace the interface motion and its velocity, a representative point with a phase-field value at is selected. Figure 4(d) compares the theoretical and predicted velocities, revealing an accuracy approaching unity after 2500 epochs of training, with a total minutes of computation time.
Figure 4. (a) Initial mesh; (b) Travelling wave solution; (c) Spacial motion of the interface; (d) Identification of the interfacial velocity; (e) Impact of external driving force; (f) PINN extrapolations. Figure 4: PINN solution for the traveling wave interface problem against the theoretical solution.
For a planar interface (no curvature), the interface velocity is expected to linearly scale with the input driving force (Eq. (25)). Figure 4(e) shows the interface velocities obtained for various driving forces, perfectly matching the theoretical expectation. Here the PINNs-MPF were first trained for a single value (Figure 4(f)) and the learning is then transferred to predict the interface velocity for various values. Such transfer of learning is typically carried out within a significantly reduced number of (maximum 100) epochs of training, as the training is not always evitable [weinan_deep_2018]. From a technical perspective, this analysis tests localized meshing, the transfer of learning and full-discrete resolution. It is worth noting that here a single neural network effectively handles the 1D scenario.
One can immediately expand the driving-force-driven interface kinetics to 2D where the planar interface is replaced by a circular interface. In this scenario, the interface has a curvature, but its effect is negligible under the condition that (Eq. (25)). Although the linear scaling of the interface velocity holds, this scenario imposes a challenge for the PINNs-MPF going from a cartesian to a spherical coordinate. PINNs-MPF predictions versus theoretical solution from OpenPhase are shown in Figure 5. Here initial trials involving a single NN (used for the 1D case above) revealed limitations marked by training instabilities and noise (c.f. the SM for qualitative and quantitative
Figure 5. (a) Initial Condition; (b) Localized mesh; (c) at Time (1 %); (d) at Time (51 %); (e) at Time (86 %); (f) Radius vs. time for different architectures. Grain prediction of PINN against theoretical and PF solutions
comparison between PINN predictions and ground truth solution for a single NN). However, stable training was obtained when applying four neural networks. Indeed, subsequent refinements increased the prediction accuracy by progressively increasing the number of neurons starting from 32, resulting in an optimal configuration of 128 neurons and 6 hidden layers, as observed in Figure 5. To quantify the differences between each PINN prediction and the theoretical solution, it is proposed in Table 4 measures of the MSE and Mean Absolute Error (MAE) for each prediction compared to the ground truth solution. The architecture with 128 neurons not only accurately matched the theoretical solution (with MAE and MSE values of and respectively) but also outperformed the phase-field solution obtained using an explicit scheme. Thus, it is hereafter proposed to keep this architecture for subsequent benchmarks as it allowed to deal with the double-well potential term and the non-convex potential term in Eq. (17) on a one hand, and to fix the related set of parameters of this architecture, especially for the multi-phase field scenario.
| Configuration | Neural Networks | HL | Neurons/HL | MAE | MSE |
| 1 | 4 | 6 | 32 | ||
| 2 | 4 | 6 | 64 | ||
| 3 | 4 | 6 | 128 |
Table 4. Quantitative comparison between each of the tested architectures and the theoretical solution.
It is worth noting that changing from the 1D to 2D setup, the PINNs-MPF is well capable of capturing and maintaining the correct interface thickness throughout the simulation.
Curvature-driven Interface Motion
Capturing the natural curvature-driven motion of an interface is the most central capability of a phase-field model, encoded into the gradient energy term and related Laplacian in the governing free energy functional and equations of motion, respectively. When the Laplacian terms take control over the interfacial motion, i.e., in Eq. (25), nonlinear kinetics arise: Here we study the shrinkage of a circular phase (grain) with , giving . This gives another level of complexity to test the PINNs-MPF framework. The number of time intervals required for the resolution of this case was 150, corresponding to , while is required to resolve the same problem using the PF scheme (c.f. the SM, section A). The results are presented in Figure 6. Here, the PINN solution fits the theoretical one. However, the phase-field linearized scheme results in a premature shrinkage of the grain. The same result is obtained even when trying with smaller time steps. This case, where the motion is fully governed by the interface and no external driving force is applied, demonstrates that PINNs can efficiently handle non-linearities in the phasefield context. Indeed, it is worth reminding that we use the tanh activation function in hidden layers to capture diverse non-linear patterns in physics-informed input data. For the output layer, the sigmoid activation ensures bounded predictions, making it suitable for interpreting outputs. The specific limits (the order parameter) remain between 0 and 1. These tailored non-linear activations allow PINN to effectively handle the grain shrinkage scenario where the motion is controlled by the interfacial energy and curvature.

Figure 6. (a) at Time (1 %); (b) Radius vs. time for ; (c) at Time (40 %); (d) Radius vs. time for ; (e) at Time (65 %); (f) Radius vs. time for . Figure 6: Grain radius prediction of PINN against theoretical and PF solutions (Open-Phase) for a natural motion. The plots correspond for PINN prediction for an interface width .
The pattern of the phase-field solution depends on the interface width ; for a thin interface (), there is a slight deviation from the theoretical solution, while the results improve with increasing the width. This behavior is systematically investigated [46], showing that it arises from the numerical limitations of calculating the Laplacian and consequently the interface curvature. This is a feature of the diffuse phase-field interfaces. The PINNs-MPF predictions were studied for three interface widths of , and . In all configurations, a consistent alignment is evident between the PINNs-MPF and the OpenPhase solutions. This is best visible for the thinnest interface shown in Figure 6b. This set of benchmarks demonstrates that the PINNs-MPF is not only capable of handling the nonlinear interface kinetics but also has a good sense of the interface width and its impact on the solution. The latter is crucial for future developments of the model to deal with interfacial phenomena, such as interfacial elasticity and solute segregation [74, 75].
Establishing Equilibrium Triple Junctions
A major capability of MPF is to handle interfacial junctions. This is not only to fulfill Young’s law but also to ensure the evolutionary path leading to a configuration with minimum energy. To test our PINNs-MPF framework, a system with four phase-fields (grains) and six triple junctions was considered. The initial mesh is visualized in Figure 7. Using our method (c.f. the SM, section C), the IC points are densified within the interfacial region, while some random points were also selected away from the interfaces to allow PINNs to learn better. To streamline computations, BC are selectively addressed along the interface lines between different batches. It is worth highlighting that the initialization having sharp angles presents a challenge for PINNs to produce accurate solutions. This is because of the curvature-driven nature of the interface dynamics, thus, initiating away from smooth curves/shapes results in complex evolutionary scenarios.
Figure 7. Initial condition points for the different NNs handling the four phases in different batches. The corresponding mesh is given in the SM, section C. Reminder: the numbers on each batch correspond to the corresponding index (cf. Figure 1). The IC points in each batch are surrounded by a circle with a common color for checking purposes.
The simulation results are visually depicted in Figure 8. Here, each grain is shown in its initial and final state of the simulation, compared to the simulation results from OpenPhase. Figure 8(m)-(Figure 8o) show the total interfacial region in the initial and final state. The comparison between PINNs-MPF and OpenPhase reveals an excellent agreement. The microstructure evolves into an equilibrium state where the angles at the triple junctions approach . One can see that the motion of the phases is well synchronized, allowing the global solution to converge to the equilibrium state. Note that the application of boundary conditions on the outer boundary of the simulation box requires the interfaces on both ends to adjust in an energy-minimizing manner.
Figure 8. Pure Phase-field solution (phase-field ) against PINN predictions for a triple-junction problem involving four grains/phases in the system.
The difference between the PINNs-MPF solution and OpenPhase lies in the interface thickness. This difference may be justified by the gradient topology of each methodology: It is indeed noted that the gradient calculation employed in the implementation, specifically using the "gradient.tape()" module in TensorFlow, differs from the Laplacian used in the regular grid computation for MPF calculations. Additional details about the difference in the computation of the Laplacian terms are given in the SM, section A. This difference explains the minor discrepancies related to the interface width. However, it can be asserted that PINNs predictions exhibit greater fidelity compared to OpenPhase when preserving similar interface widths to the initial state. We also note that, due to the curvature-driven nature of grain growth problems, handling an initialization with sharp angles () is challenging. This demonstration of the correct evolution of such a critical initialization consolidates the use of such a framework in increasingly difficult contexts. From a technical standpoint, it’s worth noting that this results in relatively slow training in the beginning until addressing the sharp angles, after which the training accelerates.
Discussion
Analyzing the training dynamics through the loss functions
The training of PINNs-MPF involves various loss terms, described through Eqs. (4) to (7). To understand and optimize the training process necessitates a comprehensive analysis of each loss in terms of their individual behavior and collective synchronization.
For the grain shrinking scenario (paragraph 4.3), the evolution of various mean losses for the four
across epochs is shown in Figure 9. Here, (i) each loss curve corresponds to the average of the four corresponding losses from the four neural networks (NNs) and (ii) the entire training process is technically conducted in a single execution. However, for the sake of clarity in illustrating the evolution of losses, we choose to present the simulation in two stages, such that, after convergence in the initial time interval, the acquired learning is then applied to restart the simulation.

Figure 9. Evolution of the different losses (PDE, IC, BC, denoising and mean loss) for the four s over the epochs for a grain shrink without driving force (for an interface width ). The plots are displayed in a logarithmic scale. In (b), values below and above are excluded from the plot for visualization purposes. Additionally, the denoising loss is omitted from the plot due to its low magnitude and fluctuating nature, which may affect the visualization of other losses.
Figure 9a shows the evolution of the different losses for the first time interval (-). The threshold is set at , marking the transition point to the next time interval. The training initially focuses on the interfacial regions, activating only the PDE and IC losses. Once the solution is computed in these regions, BC and denoising losses are subsequently activated. This aims to establish spatial continuity between batches and eliminate noise within both grain and non-grain areas, respectively. The discrete activation of BC and denoising loss has proven to be more efficient in enhancing training compared to continuous activation. It is proposed to categorize the losses into intrinsic losses, including partial PDE and IC losses, representing core elements for the NN. On the other hand, extrinsic losses, here BC and denoising, play a role in fine-tuning the solution and establishing continuity with neighboring NNs. These extrinsic losses could be subjected to optimized integration.
Figure 9(b) shows the local minimums corresponding to this transition, where all maximum losses of (as defined in Eq. (3)) fall below this threshold, indicating the onset of training for the next time intervals. The gaps between peaks correspond to the length of each time interval. The simulation could be then divided into two distinct phases. Initially, when the grain radius is substantial, the training progresses relatively fast. Subsequently, in the second phase with a smaller grain size, the BC loss exhibits an inverse relationship with the grain radius, becoming the most influential. The seamless transfer of learning between time intervals becomes evident as the PDE loss initially tends to decrease and then stabilizes in the later stages of training. Notably, the magnitudes of losses appear reasonable, with the PDE loss being approximately ten times greater than that of IC. If the BC loss remained activated throughout the entire simulation, in the subsequent benchmark (triple junction), it was managed through discrete activation, mirroring the approach adopted during the initial time interval.
For the triple junction scenario (paragraph 4.4), the evolution of various losses (PDE, IC, BC, and mean loss) for the 16 NNs across epochs is illustrated in Figure 10. In this context, the threshold is now set at .
(a) (a) Training on the first time interval - using the pyramidal approach.
(b) (b) Training on the first time interval - without the pyramidal approach.
(c) (c) Training on the whole time domain.
Figure 10. Evolution of the different losses for the 16 s over the epochs for the triple junction scenario. The plot is displayed in a logarithmic scale.
Similar to the grain shrinking scenario, the first time interval is addressed, and then the cumulative learning is used to restart the simulation. For this purpose, two approaches are compared, with and without a pyramidal approach (respectively in Figure 10a and Figure 10b). It is noted that the only difference between the two simulations is the number of batches for the fine subdivision (c.f. Algorithm 1). Specifically, and for the pyramidal approach, while for the second configuration. In Figure 10a, the impact of the optimization techniques is subsequently showcased: wheel of optimizers and the pyramidal training. The wheel of optimizers is activated with a period of 50 epochs; L-BFGS-B initially decreases the global loss from a magnitude of to MSE, a progressive decrease is achieved with subsequent rounds to below the threshold of MSE. Then, Adam allows the transfer of learning from level 1 to level 2 of the pyramid, where the whole spatial domain is handled. Even if the global loss increases initially, it quickly falls below the target threshold (the 16 NNs converge together within 850 epochs), demonstrating the model’s capacity to generalize the learnable solution to the whole domain. However, the non-adoption of such a strategy makes it hard to enforce the convergence of all the NNs, as seen in Figure 10b, and the global loss stagnates above the threshold even after 6000 epochs of
training, using the same set of parameters as in Figure 10a except for the non-activation of the pyramidal training. It is worth noting that such a comparison was subjected to excessive re-execution for repeatability with the same outputs; the 16 NNs failed to converge simultaneously without the activation of pyramidal training during the first time interval. It is also suggested that this finding is also attributed to two main impacts of pyramidal training: the progressive data augmentation basis of this concept, and the facilitation of addressing boundary conditions, as accurate boundary predictions are a direct consequence of good convergence within the domain. Additional details about the domain decomposition related to the pyramidal approach are provided in the SM, section D.
The black dashed line in Figure 10c corresponds to the total mean loss, exhibiting oscillations that dip below the threshold (red line) for the enhancement of the next training interval. Here, the boundary condition loss is not visualized due to its discrete activation; however, its magnitude is noted to be in the range of . It is noteworthy that all losses for all NNs should fall below this limit (not just the mean values). Previous observations regarding the behavior of the IC and PDE losses remain relatively valid in terms of the stabilization of losses across epochs. However, the declining aspect observed in the previous scenario is not evident here. This discrepancy could be attributed to the complexity of the triple junction evolution and dynamic interactions between different phases when compared to the single-grain shrinkage scenario. The sum loss (Eq. (16)) has the lowest magnitude (), and its constant values reflect the algorithm’s capacity to dynamically correct predictions, ensuring adherence to the summation constraint.
Addressing Nonlinearities in Microstructure Prediction
The current implementation showcases the effectiveness of this method in addressing non-linearities such as in the MPF model, offering good predictions of microstructure evolution in different scenarios above. The ability to capture both curvature-driven and driving-force-driven interface dynamics is well demonstrated. Typical MPF simulations employ a time-stepping approach with a magnitude of up to and a minimum of 1000 steps across the given scenarios that is to ensure computational stability. In contrast, applying PINNs is shown here to allow for the selection of much larger time-stepping up to 200 time steps, except for the traveling interface simulation where computational speed was a focal consideration. This difference stems from the fact that while the conventional MPF is limited due to the spatial resolution, the PINNs-MPF focuses on the overall microstructural patterns, substantiating its ability to capture underlying physics while maintaining stable computational schemes.
Tuning the model hyper-parameters
One of the crucial points when dealing with PINNs is the tuning of the hyper-parameters. Here, any simplification made for hyper-parameters directly impacted convergence and can reduce the efforts for trial and error. In terms of the training dataset, this is automatically set, simplifying user inputs to the initialization, physical, and geometrical parameters. On the other hand, we have adopted major concepts from conventional MPF frameworks, namely the OpenPhase [46]. This includes considering our simulation domain heterogeneously, with a focus on the phase-field interfacial region, while parallelizing the handle of individual phase-field variables. This demonstrates that the PINNs-MPF can successfully inherit optimization techniques already applied in classical computing approaches. Especially, the PINN resolution can be then assimilated as a multi-variable sequential learning, showcasing its adaptability and integration capabilities. The application of multi-networking and training by blocks demonstrates the feasibility of addressing diffuse interface problems with simplified hyper-parameters.
For specific challenges handling moving boundaries, the implementation of adaptive weights (gradient scaling algorithm) was unavoidable [32]. It is therefore worth noting that the loss terms in the implemented PINNs-MPF are by default left unweighted, i.e. in Eq. (3). This reduces the complexity of the current framework and the need for meticulous tuning, especially when considering the diverse nature of loss terms in Eq. (3). Moreover, this kind of algorithm could be useful to integrate in more complex scenarios. We note that our numerical experiments have indicated that the extrinsic losses can be weighted to optimize performance; for instance, reducing the weights for the BC loss to expedite training, and conversely, increasing the weighting of the phase summation loss to allocate more attention to it. However, the intrinsic losses (IC and PDE) cannot be weighted.
Interfacial Attention for a Latent Microstructure Representation
The results of the conducted benchmark studies indicate that the global PINN solution in simulations involves a precise computation of solutions along interface regions throughout the entire domain, complemented by an extrapolation method to approximate solutions at the interface region. This hybrid approach proves beneficial for diffuse-interface problems using ML. In the triple junction scenario, the distributed ’0 and 1’ points, as depicted in Figure 7, were employed to enhance the precision of the solution, particularly for the application of phase summation (Eq. (2)) and accurate addressing of boundary conditions between neighboring NNs. Similarly to the mono-phase benchmarks, it is possible to fully focus on interfacial regions within multi-phase simulations. First demonstrations of this are provided within Section E of the SM. This opens the way to resolve MPF simulations using NNs independently of the grid size, thus allowing for tackling more complex scenarios.
Dynamic correction mechanism for phase summation constraint
Within the literature, a major limitation against the application of ML to resolve multi-phase problems is the phase summation criterion: Indeed, in previous works, a requirement for a bounding algorithm was obligatory as a correction step to the PINN prediction [36, 39]. Meanwhile, in the current work, the applied method introduces a dynamic correction mechanism for phase predictions. This involves dividing the prediction of each neural network (NN) responsible for a specific phase by the sum of predictions from other NNs handling other phases within the same batch. This dynamic correction not only ensures continuous training but also suppresses the need for an external correction algorithm. Instead, the correction of phase summation is seamlessly integrated as an additional constraint loss term, enhancing the efficiency of the training process.
Exploring model complexity and impact of the optimization strategy
The results about the additional numeric experiment are gathered in Table 5. Associated graphical illustration is provided in Figure 11. The experiments revealed that while all architectures started with high and similar MSE values, the MSE decreased significantly after the first round of Adam optimization. The four models reach similar loss magnitude after the first round. Subsequent L-BFGS optimization further reduced the MSE for the and architectures, with the model attaining the best overall MSE of . Larger architectures like and failed to converge within the specified rounds, potentially due to overfitting or optimization challenges arising from their high parameter counts. The architecture struck an optimal balance between model complexity and generalization ability, benefiting from the combined Adam and L-BFGS optimization strategy. While larger models had more parameters, they did not necessarily perform better and could be computationally expensive. This numerical experiment reveals a solid correlation between the model’s parameter count, representing its complexity, and the impact of multi-networking. For instance, the configuration, yielding 1,651,205 trainable parameters, challenges training when employing four NNs. Conversely, a reduced parameter count of 1,414,417 enables training 16 NNs (16 x (6 HL x 128 neurons)) simultaneously to effectively handle four phases (c.f. the triple junction benchmark). Moreover, even higher parallelism is feasible through pyramidal training, where initially 64 NNs are initialized, but only 16 are selected for further training using the basket of PINNs concept. That implies that using moderately-sized networks through the multi-networking context (the extended subdomain decomposition technique) allows to efficiently alternate optimizers (the wheel of optimizers), while the transfer of learning facilitates the transition between time intervals. This underscores the rationale behind initially selecting a subset of PINN techniques, while the second subset (AMFO, phase correction, and vertical optimization) has proven crucial in prior MPF method studies and the current work. Pyramidal training and the comprehensive set of PINNs emerge as essential for progressively transferring learning to a unified domain when dealing with massive domain decomposition, thereby making the framework ready for up scaling, as hereafter detailed.
| HL neurons/HL | Trainable parameters | MSE at epoch 0 | MSE after round 1 | MSE after round 2 | Rounds required to converge (*) |
| 6 x 64 | 105,605 | 5 | |||
| 6 x 128 | 416,005 | 3 | |||
| 6 x 256 | 1,651,205 | ||||
| 6 x 512 | 6,579,205 |
Table 5. Evolution of MSE and model trainable parameters count for different architectures: optimization rounds with Adam-L-BFGS-Adam cycles. Comparisons are done at epoch 0, after first and second rounds. (*) It is worth reminding that convergence signifies that the global loss of the model has fallen below the threshold and the current time interval has been managed.
Figure 11. Evolution of MSEs and model parameters count for different architectures as variants of the benchmark 2.
Transfer of learning
In the context of learning transfer, our framework demonstrates the feasibility of hierarchically transferring knowledge within subdomain decomposition. Through multi-networking and full parallel training, it employs a pyramidal training approach, categorized as ’Multiple to Multiple’ transfer. This allows the transfer of learning, for example (but not limited to), from 4 NNs to 4 others, 16 to 16, 16 to 4, etc. From a technical standpoint, achieving ’Multiple to Single’ transfer of learning is possible, where knowledge is transferred from multiple NNs to a single NN and vice versa. However, our first trials showcased that training a single NN revealed challenges, leading to optimization failures. The synergy between ML and optimization algorithms is crucial for effectively exploiting patterns, but this task becomes challenging with large NNs. Multi-networking and pyramidal training until reaching a reasonable number of NNs proves efficient in managing such complexities. Nevertheless, we provide within the actual code repository a demonstration of this ’Multiple to Single’ transfer, denoted as ’ to Master’ transfer, that could be of interest for other physical applications. Additional details are provided in the SM, section F.
Exploring more ways for improvement of the framework
In the current development phase, one aspect to consider is selecting the appropriate threshold to transition between time intervals in the full-discrete resolution. This threshold was found to be strictly related to each simulation and subjected to trial and error. For the magnitude of in single-phase benchmarks, it was slightly raised when handling the multiphase field (MPF) scenario to ensure a compromise between the speed of the training and the convergence of the solution. One possibility to deal with this limitation is to integrate an additional loss term related to the variation in the energy of the system. If the energy residual (and its derivative) reaches a steady state, it is then a physical indication that the system reaches its equilibrium within this time interval, and then the transition could be ensured [76, 77]. Theoretical formulations and graphical illustrations related to the utility of integrating energy loss to enhance training are provided in the SM, section B.
A potential point of improvement for the current PINNs-MPF framework is the integration of sequential training. Indeed, optimizing the evolution of losses related to the triple junction motion could be achieved if the model captures the temporal behavior of the solution. The flexibility of discrete resolution under the current implementation eases the integration of sequential learning through Recurrent Neural Networks. Optimizing runtimes against theoretical solutions or existing literature is beyond the scope of this study, as PINNs are inherently more time-consuming than classical approaches. Enhancing PINN solvers is a work in progress [2, 78] and studies on the use of PINNs for solving multi-phase problems remain uncommon [32]. Therefore, the first goal of this study is to prioritize solution fidelity, training continuity, and efficient resource distribution. This approach is similar to recent studies focusing on multiphase phenomena, particularly in fluid dynamics. For instance, Haghighat et al. [32] have concentrated on the methodology and challenges associated with training PINNs for coupled flow and deformation in porous media. Zheng et al. [36] have aimed to predict the sequential motions of flow simulations in a discontinuous manner, alternating between PINN training and MCBOM corrections. Amini et al. [34] have focused on validating the inverse modeling approach targeting nonisothermal multiphase poromechanics. To conduct running time studies on these multi-phase field frameworks, inspiration could be taken from the study of Shukla et al. [62]. Among other proposals, leveraging multiple GPUs and nodes and adopting a hybrid programming model could be beneficial.
Conclusion
An interconnected PINNs-MPF framework has been presented to resolve MPF simulation of interface dynamics. Our approach has been to address and benchmark most central requirements for a reliable reproduction of MPF simulations, namely, curvature- and driving-force-driven interface motions and the evolution of a triple junction. In particular, the nonlinear evolution of curved interfaces and the effect of interface width on the interface kinetic were captured, demonstrating the capability of this framework in dealing with diffuse interfaces. The simulation of the triple junction established that the application of the sum constraint as a loss term and in combination with a parallelization scheme works precisely and efficiently. The rationale behind the current development has been to demonstrate the feasibility of PINNs in capturing key features of microstructure evolution, before increasing complexities. This ensures a well-prepared transition to more advanced dimensions while maintaining the robustness and reliability of the methodology. All benchmark simulations are conducted in a single execution without the need for post-correction algorithms, such as intercalated phase correction algorithms. Various advanced techniques were encompassed in the PINNs-MPF framework, including training optimization, extended domain decomposition, and boundary condition propagation. These techniques have yielded accurate predictions and efficient training, establishing a robust foundation for future advancements in this research area. Potential avenues for further exploration include developing more unified solutions through sequential learning, conducting comparative studies with other PINN approaches in the literature, and integrating energy-based optimizations, which is a key component of the MPF Method. As this progress step, the vision is to extend the capability of the developed framework to address more intricate scenarios involving a greater number of phases, larger grids, and three-dimensional dimensions. As the difficulty of the scenarios gradually increases, it is crucial to carefully address related issues when using machine learning to predict solutions, such as managing singularities, mitigating inaccurate predictions, and optimizing computation times.
Acknowledgments
RDK acknowledges financial support from the German Research Foundation (DFG) within projects DA 1655/2-1 (Heisenberg program) and DA 1655/3-1.
Supplementary material
Additional results and discussions as well as a video presentation of the framework are given in the Supplementary Material attached.
Data availability
The PINNs-MPF repository is available on GitHub at the following link:
https://github.com/SFETNI/PINNs_MPF
Competing financial interests
The authors declare no competing financial interests.
Specific terminologies of PINNs-MPF
As specific terminologies are introduced in this work, it is important to remind the definition of each one as follows.
Extended domain decomposition: compared to the domain decomposition proposed in the literature (e.g., XPINN, PPINN), the current framework introduces an extension to handle the phase domain for Multi-Phase Field (MPF) simulations, as well as a specific synchronisation method to handle the phases domain. c.f. the related paper for more details.
Discrete resolution: we propose to keep this terminology to be aligned with the MPF Method and classical approaches. Regarding the literature of PINNs, this approach makes our framework classed among the time-marching PINNs/temporal PINNs (causality literature).
Wheel of optimizers: this reflects the training optimization approach by alternating two optimizers, Adam and L-BFGS, in a cyclic way to leverage their respective strengths for improved convergence.
Round of optimization: a L-BFGS optimization.
Basket of PINNs: training multiple neural networks simultaneously can be computationally expensive. This concept is technically implemented through the dynamic selection of networks that require further optimization at each L-BFGS round. In the case of pyramidal training, a neural network is selected from each block independently of the level of subdivision, leading to the simultaneous training of up to 16 neural networks for the triple junction benchmark (instead of 64 NNs).
Pyramidal training: involves gradually subdividing the spatial domain, for each phase/grain, into finer blocks and training selected NNs at the finest level first. The training then transitions to coarser levels, with each selected PINN receiving a warm start from the previously trained PINN at the finer level.
Adaptive Mesh-Free Optimization (AMFO): is an extreme mesh refinement process adapted from the Adaptive Mesh Refinement (AMR) technique in the phase-field and MPF literature, incorporating dynamic resampling and denoising processes..
Horizontal optimization: refers to the AMFO.
Vertical optimization: refer to the optimization of interactions between phases.
Pure Multi-Phase-Field resolution
The Gibbs free energy of a multi-phase-field (MPF) system could be seen (from a first approach) as a summation of an interfacial free energy density (also called Grain-Boundary (GB)) and an external one as follows.
The other contributions could be added added to the MPF free energy formulation, such as the mechanical, chemical and magnetic ones.
is the non-conservative phase-field variable corresponding to the phase (idem for ), N is the number of components/crystals present in the system, is the interfacial energy while is the interface width.
For a system with two-phases (binary), the previous expression becomes:
or simply:
Th temporal evolution of the non-conserved order parameter follows the Allen-Cahn formulation:
in 1D, for a dual interface, when considering only the interfacial energy , the AC formulation gives:
This expression could be also expressed as:
where (for more details about the identification of .)
We should note that in the previous equation, only the interface contribution was considered. However, in order to guarantee a moving interface, the non-interfacial contributions should be taken into account:
The evolution of the order parameter with respect to time is described by this equation:
where the left side describes the interfacial changes while the right one controls the moving of the interface. To solve this equation, we can use the following time-stepping scheme:
then
where and are the Fourier coefficients of at time steps and respectively, is the time step size, is the wave number, and is a constant parameter. The resolution could be performed using or not FFT. Hereafter using en Euler scheme:
Then
Here, we note that the Laplacian is applied on a regular periodic grid:
However, for the resolution of the same problem using PINN, another methodology is applied as follows, once the computing is applied on batches and not on a periodic grid.
Algorithm A.1: Compute Laplacian using PINN
Data: Input parameters: x, y, t
Result: Compute Laplacian using PINN
1 Initialize persistent gradient tape;
/* Watch variables for differentiation */
/* Concatenate input variables */
2 g ← concatenate(x, y, t);
/* Evaluate phase field at current location */
3 ϕα ← evaluate(g);
4 ϕα ← clip(ϕα, 0.0, 1.0);
/* Watch the phase field for differentiation */
// Compute gradients
5 ϕtα ← tape.gradient(ϕα, t);
6 ϕxα ← tape.gradient(ϕα, x);
// Watch the phase field for differentiation
7 ϕyα ← tape.gradient(ϕα, y);
// Watch the phase field for differentiation
8 ϕxxα ← tape.gradient(ϕxα, x);
9 ϕyyα ← tape.gradient(ϕyα, y);
/* Compute Laplacian */
10 lap(ϕα) ← ϕxxα + ϕyyα;
Energy controlled problem
In this section, we showcase that the energy residual could be also integrated as an additional loss function and be an alternative to the threshold to move from one time interval to another. If we consider that is constant and depends on , then the expression for the energy evolution of the system at each time is given by:
Through this equation, one can differentiate two different energy residuals that should compensate each other during the motion: a term related to the energy derivative while the second one is related to the order parameters. We note that the negative sign in the energy values is only numerical and reflects the shrinking nature of the grain (in the grain shrink scenario).
Mesh methodology
In this section, we illustrate the collocation points generated to simulate the motion of the triple junction. As shown in Figure C.1, the meshing approach prioritizes interfacial points, neglecting areas outside the grains. The machine learning solution is computed within interfacial regions, while approximations are made for regions outside the grains.
Common batches In the current implementation, 16 neural networks () are trained, implying 16 sets of collocation points. To optimize training and reduce computational costs, these 16 sets are reduced to 4 as follows: The instances handling the same batch are assigned a common batch. This common batch is constructed based on the initial condition (IC) set of each , as illustrated in Figure C.2. This approach operates under the assumption that if two phases/grains are in interaction, this occurs within interfacial regions. Consequently, these two phases are assigned a common batch, grouping their interfacial points. This process is then generalized to all interacting phases/grains within the same batch.

Figure B.1. Evolution of the energy loss terms.
Pyramidal training
The pyramidal training is illustrated in Figures D.1 and D.2 respectively. The pyramidal approach functions as a preliminary training step, characterized by a meticulous subdivision into 16 batches within each phase domain.
Block training Starting with Figure D.1, the bottom level marks the initiation of the pyramidal approach. To optimize training efficiency and avoid deploying 64 NNs/s for the entire set of batches, here a progressive training is then adopted. Specifically, only highlighted batches from each Quarter/Block are trained in the initial round. This reduces the number of NNs to train to 16, offering the model an opportunity to focus on a smaller spatial area. This enhances the model’s ability to capture features and dependencies between outputs and inputs. In this methodology, Block 0 serves as a reference, where the batch with the highest number of interfacial points is chosen for the pre-training process. For subsequent blocks, the selection criterion involves choosing each with the same batch number. This ensures consistency in the application of the concept of common batches. We note that the pyramidal training was conducted by meshing only the interfacial zones. Our experiments have indeed shown that incorporating additional points outside these areas proved to be beneficial in constraining the summation of the phases at advanced stages of the training. Advancing to the upper level of training (Figure D.2), each pre-trained from the lower level (fine subdivision) transfers its learning to the corresponding within the same Block in the coarser subdivision. This progressive data augmentation has proven to be efficient in enhancing the training process.
Toward enhanced Multi-Phase Field simulations: full focus on interfacial regions for scalable grid handling
First results for the triple junction without full attention to interfacial regions are available in Figure E.1. In the main benchmark of the associated manuscript, grain/no-grain regions were assigned a minimum number of IC points. This helps the model make accurate phase summation corrections on one hand, and efficiently handle the boundary interactions of neighboring networks on the other. Applying the denoising loss, it is possible to handle the simulation with full focus on interfacial regions. However, our initial trials have shown that the grain/no-grain regions require implicit consideration and specific modifications within the code. These modifications concern functions handling the boundary interactions between NNs to incorporate consideration for the boundary prediction of the grain/no-grain lines, albeit implicitly (without training data). Once achieved, the grid size can be significantly increased, allowing for the exploration of more complex microstructures and the addition of a third dimension, thereby enhancing the dimensionality of the problem.

Figure C.1. Sets of the collocation points for the different NNs handling the four phases in different batches.
to Master learning transfer
The s to Master learning transfer is illustrated in Figure F.1 and can also be referred to as Master to since this transfer is circular. The knowledge accumulated over the entire domain can be redistributed to all NNs, enabling the exploration of similar physical problems. The learning is transferred between levels using a pyramidal approach. The base of the pyramid is set to 64, representing 64 subdivisions. The subdivision is then progressively increased until reaching 4 NNs. At this level, once the 4NNs are trained, one selected NN with the minimum loss is chosen as a candidate to handle the entire domain and becomes a Master . This approach allows for continuous coverage of the entire domain. However, as previously reported in the Discussion section of the related manuscript, our experiments highlighted difficulties in optimizing weights, particularly using the LBFGS optimizer, due to the complexity of the problem for a single NN. Handling the upper level using 4 NNs, interacting at the boundaries, was found to be very efficient. In the technical demonstration found in the GitHub repository (link in the main paper), a demonstration with up to 16 batches/phase (64 initial s) is successfully benchmarked for the simulation of the triple junction.
First findings about training using a single 1NN and the requirement for networking
Figure F.2 a PINN solution without multi-networking. This solution, configured with 1 NN, 6 Hidden Layers (HL), and 128 neurons per HL, without domain decomposition (employing one Network), illustrates a significant discrepancy between the prediction and the theoretical model. This discrepancy persists despite the simplified conditions compared to the benchmark 2 of grain shrinking (dimensionless grain radius of 0.3 instead of 0.4 and a driving force of -100 instead of -250). This highlights the necessity for domain decomposition to achieve accurate solutions for this physical problem.

Figure C.2. Illustration of the common batch 3: In this example, the instances 0300, 0301, 0302, and 0303, each handling phases 0, 1, 2, and 3, respectively, are assigned a shared set of collocation points. This operation is concurrently performed for other batches. We highlight that out-interface points are not considered during the pyramidal training.

Figure D.1. Bottom level of the pyramidal approach and selected batches for the “pre-training”. In Block 0, the (Batch 05 - Phase 0) is chosen. In subsequent phases within Block 0, 0501, 0502, and 0503, are selected, ensuring they share the same batch number as , in order to systematically construct the common batches as shown in Figure C.2. This selection process is consistently applied to other Blocks as well. Consequently, a set of 16 is selected and placed in the Basket.

Figure D.2. Upper level of the pyramidal approach: the training is conducted within the full spatial domain for each Block. Subsequently, continuity between boundaries is addressed. The training is then warm-started, wherein each receives the weights (and thus accumulates learning) from the selected during the pre-training phase. For example, in Block 0, (responsible for phase 1 in batch 0) receives the weights from .

(a) IC Phase 0

(b) Phase 0

(c) IC Phase 1

(d) Phase 1

(e) IC Phase 2

(f) Phase 2

(g) IC Phase 3

(h) Phase 3
Figure E.1. State of the triple junction at approximately the middle of the simulation (), with full attention to interfacial areas (out-interface regions are excluded). Here, we note that the code, in its current state, allocates a minimum number of points for a batch where there are no interfacial points (as in batches 0 and 2 of phase 1). Additionally, the denoising loss plays a crucial role in addressing corrections for the extrapolations of the solution to the grain/no-grain regions. This benchmark is available within the GitHub repository (link in the main paper).

Figure F.1. Illustration of the ’Multiple to Master’ and ’Master to Multiple’ Transfer of the learning. In this illustrative example (could be also called ”Master to s - s to Master” pyramid), the base consists of 64 batches, with each batch containing one for training. Each level is divided into four quarters or blocks. For each quarter batch, only the s containing interfacial points are pre-selected. Then, a given (for example, in batch 22) is selected to transfer its learning to the subsequent level within the same quarter; this selection could be based on the percentage of interfacial points, or the with the minimum loss after training the whole set of . For the next level (36 batches), all s within each quarter receive learning from the selected from the previous level within the same quarter. Then, the same principle of preselection occurs; for example, the with batch 10 is selected. This mechanism is repeated until reaching the upper level of the pyramid. Here, the selected will be trained not only in its batch but within the global domain, serving as the Master PINN. In the technical demonstration found in the GitHub repository (link in the main paper), a demonstration with up to 16 batches (s) is successfully benchmarked for the simulation of the triple junction.

Figure F.2. Presenting to the reader a PINN solution without multi-networking in the supplementary material with reference to this in the article.
References
- [1]Kamal Choudhary et al. “Recent advances and applications of deep learning methods in materials science”. In: npj Computational Materials 8.1 (). ISSN: 2057-3960. DOI: 10.1038/s41524-022-00734-6.DOI
- [2]Salvatore Cuomo et al. “Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next”. In: Journal of Scientific Computing 92.3 (2022), p. 88. ISSN: 0885-7474. DOI: 10.1007/s10915-022-01939-z.DOI
- [3]Ehsan Haghhighat et al. “A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics”. In: Computer Methods in Applied Mechanics and Engineering 379 (2021), p. 113741. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2021.113741.
- [4]P. Ramuhalli, L. Udpa, and S.S. Udpa. “Finite-element neural networks for solving differential equations”. In: IEEE Transactions on Neural Networks 16.6 (2005), pp. 1381–1392. DOI: 10.1109/TNN.2005.857945.DOI
- [5]M. Raissi, P. Perdikaris, and G.E. Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In: Journal of Computational Physics 378 (Feb. 2019), pp. 686–707. ISSN: 00219991. DOI: 10.1016/j.jcp.2018.10.045. URL: https://linkinghub.elsevier.com/retrieve/pii/S0021999118307125.DOI
- [6]Yibo Yang and Paris Perdikaris. “Adversarial uncertainty quantification in physics-informed neural networks”. In: Journal of Computational Physics 394 (2019), pp. 136–152. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2019.05.027. URL: https://www.sciencedirect.com/science/article/pii/S0021999119303584.
- [7]Dongkun Zhang et al. “Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems”. In: Journal of Computational Physics 397 (2019), p. 108850. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2019.07.048.
- [8]Michelle Tindall Prakhar Sharma Lion Evans and Perumal Nithiarasu. “Hyperparameter selection for physics-informed neural networks (PINNs) – Application to discontinuous heat conduction problems”. In: Numerical Heat Transfer, Part B: Fundamentals 0.0 (2023), pp. 1–15. DOI: 10.1080/10407790.2023.2264489.DOI
- [9]“A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications”. In: Engineering Applications of Artificial Intelligence 101 (2021), p. 104232. ISSN: 0952-1976. DOI: https://doi.org/10.1016/j.engappai.2021.104232.
- [10]Yu Diao et al. “Solving multi-material problems in solid mechanics using physics-informed neural networks based on domain decomposition technology”. In: Computer Methods in Applied Mechanics and Engineering 413 (2023), p. 116120. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2023.116120. URL: https://www.sciencedirect.com/science/article/pii/S004578252300244X.
- [11]Alexander Henkes, Henning Wessels, and Rolf Mahnken. “Physics informed neural networks for continuum micromechanics”. In: Computer Methods in Applied Mechanics and Engineering 393 (2022), p. 114790. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2022.114790. URL: https://www.sciencedirect.com/science/article/pii/S0045782522001268.
- [12]Xin-Yu Guo and Sheng-En Fang. “Structural parameter identification using physics-informed neural networks”. In: Measurement 220 (2023), p. 113334. ISSN: 0263-2241. DOI: https://doi.org/10.1016/j.measurement.2023.113334.
- [13]Jinshuai Bai et al. An introduction to programming Physics-Informed Neural Network-based computational solid mechanics. 2023. arXiv: 2210.09060 [cs.CE].arxiv.org/abs/2210.09060
- [14]Jinshuai Bai et al. “An Introduction to Programming Physics-Informed Neural Network-Based Computational Solid Mechanics”. In: International Journal of Computational Methods 20.10 (2023), p. 2350013. DOI: 10.1142/S0219876223500135.DOI
- [15]Somdatta Goswami et al. “Transfer learning enhanced physics informed neural network for phase-field modeling of fracture”. In: Theoretical and Applied Fracture Mechanics 106 (2020), p. 102447. ISSN: 0167-8442. DOI: https://doi.org/10.1016/j.tafmec.2019.102447. URL: https://www.sciencedirect.com/science/article/pii/S016784421930357X.
- [16]M.S. Khorrami et al. “An artificial neural network for surrogate modeling of stress fields in viscoplastic polycrystalline materials”. In: npj Comput Mater 9 (2023), p. 37. DOI: 10.1038/s41524-023-00991-z.DOI
- [17]Yuli Cao, Ruina Xu, and Peixue Jiang. “Physics-informed machine learning based RANS turbulence modeling convection heat transfer of supercritical pressure fluid”. In: International Journal of Heat and Mass Transfer 201 (2023), p. 123622. ISSN: 0017-9310. DOI: https://doi.org/10.1016/j.ijheatmasstransfer.2022.123622.
- [18]Shirui Luo et al. “Parameter Identification of RANS Turbulence Model Using Physics-Embedded Neural Network”. In: High Performance Computing. Ed. by Heike Jagode et al. Cham: Springer International Publishing, 2020, pp. 137–149.DOI
- [19]S. Hanrahan, M. Kozul, and R.D. Sandberg. “Studying turbulent flows with physics-informed neural networks and sparse data”. In: International Journal of Heat and Fluid Flow 104 (2023), p. 109232. ISSN: 0142-727X. DOI: https://doi.org/10.1016/j.ijheatfluidflow.2023.109232. URL: https://www.sciencedirect.com/science/article/pii/S0142727X23001315.
- [20]Haotian Chen et al. “A Critical Evaluation of Using Physics-Informed Neural Networks for Simulating Voltammetry: Strengths, Weaknesses and Best Practices”. In: Journal of Electroanalytical Chemistry 925 (2022), p. 116918. ISSN: 1572-6657. DOI: https://doi.org/10.1016/j.jelechem.2022.116918.
- [21]Tobias Hofmann et al. “Physics-Informed Neural Networks for State of Health Estimation in Lithium-Ion Batteries”. In: Journal of The Electrochemical Society 170.9 (Sept. 2023), p. 090524. DOI: 10.1149/1945-7111/acf0ef.DOI
- [22]Jiang-Xia Han et al. “Physics-informed neural network-based petroleum reservoir simulation with sparse data using domain decomposition”. In: Petroleum Science 20.6 (2023), pp. 3450–3460. ISSN: 1995-8226. DOI: https://doi.org/10.1016/j.petsci.2023.10.019. URL: https://www.sciencedirect.com/science/article/pii/S1995822623002947.
- [23]Gaurav Priyadarshi et al. “Parametric investigation and optimization of phase change material-based thermal energy storage integrated desiccant coated energy exchanger through physics informed neural networks oriented deep learning approach”. In: Journal of Energy Storage 80 (2024), p. 110231. ISSN: 2352-152X. DOI: https://doi.org/10.1016/j.est.2023.110231. URL: https://www.sciencedirect.com/science/article/pii/S2352152X23036307.
- [24]Ameya D. Jagtap and George Em Karniadakis. “Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations”. In: Communications in Computational Physics 28.5 (2020), pp. 2002–2041. DOI: https://doi.org/10.4208/cicp.UA-2020-0164.
- [25]Ehsan Kharazmi, Zhongqiang Zhang, and George E.M. Karniadakis. “hp-VPINNs: Variational physics-informed neural networks with domain decomposition”. In: Computer Methods in Applied Mechanics and Engineering 374 (2021), p. 113547. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2020.113547.
- [26]Xuhui Meng et al. “PPINN: Parareal physics-informed neural network for time-dependent PDEs”. In: Computer Methods in Applied Mechanics and Engineering 370 (2020), p. 113250. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2020.113250. URL: https://www.sciencedirect.com/science/article/pii/S0045782520304357.
- [27]Michael Penwarden et al. “A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions”. In: Journal of Computational Physics 493 (2023), p. 112464. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2023.112464.
- [28]A. Krishnapriyan et al. “Characterizing possible failure modes in physics-informed neural networks”. In: Advances in Neural Information Processing Systems. Vol. 34. 2021, pp. 26548–26560.arxiv.org/abs/2109.01050
- [29]Alex Bilho and Roman O. Popovych. “Physics-informed neural networks for the shallow-water equations on the sphere”. In: Journal of Computational Physics 456 (2022), p. 111024. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2022.111024.
- [30]Carlos J.G. Rojas, Jos L. Boldrini, and Marco L. Bittencourt. “Parameter identification for a damage phase field model using a physics-informed neural network”. In: Theoretical and Applied Mechanics Letters 13.3 (2023), p. 100450. ISSN: 2095-0349. DOI: https://doi.org/10.1016/j.taml.2023.100450.
- [31]Wen Zhang and Jian Li. “The robust physics-informed neural networks for a typical fourth-order phase field model”. In: Computers & Mathematics with Applications 140 (2023), pp. 64–77. ISSN: 0898-1221. DOI: https://doi.org/10.1016/j.camwa.2023.03.016.
- [32]Ehsan Haghighat, Danial Amini, and Ruben Juanes. “Physics-informed neural network simulation of multiphase poroelasticity using stress-split sequential training”. In: Computer Methods in Applied Mechanics and Engineering 397 (2022), p. 115141. ISSN: 0045-7825.DOI
- [33]Zhao Zhang et al. “A physics-informed convolutional neural network for the simulation and prediction of two-phase Darcy flows in heterogeneous porous media”. In: Journal of Computational Physics 477 (2023), p. 111919. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2023.111919.
- [34]Danial Amini, Ehsan Haghighat, and Ruben Juanes. “Inverse modeling of nonisothermal multiphase poromechanics using physics-informed neural networks”. In: Journal of Computational Physics 490 (2023), p. 112323. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2023.112323.
- [35]Colby L. Wight and Jia Zhao. Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks. 2020. arXiv: 2007.04542 [math.NA].arxiv.org/abs/2007.04542
- [36]Haoyang Zheng, Ziyang Huang, and Guang Lin. “A physics-constrained neural network for multiphase flows”. In: Physics of Fluids 34.10 (Oct. 2022). DOI: 10.1063/5.0111275.DOI
- [37]Yinhao Zhu et al. “Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data”. In: Journal of Computational Physics 394 (2019), pp. 56–81. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2019.05.024.
- [38]Amirhossein Arzani, Kevin W. Cassel, and Roshan M. D’Souza. “Theory-guided physics-informed neural networks for boundary layer problems with singular perturbation”. In: Journal of Computational Physics 473 (2023), p. 111768. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2022.111768.
- [39]Ziyang Huang, Guang Lin, and Arezoo M. Ardekani. “A consistent and conservative volume distribution algorithm and its applications to multiphase flows using Phase-Field models”. In: International Journal of Multiphase Flow 142 (2021), p. 103727. ISSN: 0301-9322. DOI: https://doi.org/10.1016/j.ijmultiphaseflow.2021.103727.
- [40]Long-Qing Chen. “Phase-field models for microstructure evolution”. In: Annual Review of Materials Research 32 (2002), pp. 113–140.DOI
- [41]Ingo Steinbach. “Phase-field models in materials science”. In: Modelling and simulation in materials science and engineering 17 (2009), p. 073001.DOI
- [42]Ingo Steinbach. “Phase-field models in materials science”. In: Modelling and Simulation in Materials Science and Engineering 17.7 (2009). ISSN: 09650393. DOI: 10.1088/0965-0393/17/7/073001.DOI
- [43]Christopher J. Grose and Paul D. Asimow. “A multi-phase field model for mesoscopic interface dynamics with large bulk driving forces”. In: Computational Materials Science 212 (2022), p. 111570. ISSN: 0927-0256. DOI: https://doi.org/10.1016/j.commatsci.2022.111570.
- [44]Avtar Singh and Siladitya Pal. “Multi-phase field modeling for various fracture mechanisms in composites”. In: Engineering Fracture Mechanics 241 (2021), p. 107348. ISSN: 0013-7944. DOI: https://doi.org/10.1016/j.engfracmech.2020.107348. URL: https://www.sciencedirect.com/science/article/pii/S0013794420309292.
- [45]L Huo et al. “Multi-phase field modeling and simulation of magnetically driven grain boundary migration in SmCo polycrystals”. In: Journal of Physics D: Applied Physics 56.46 (Aug. 2023), p. 465003. DOI: 10.1088/1361-6463/acedbb. URL: https://dx.doi.org/10.1088/1361-6463/acedbb.
- [46]Reza Darvishi Kamachali. “Grain boundary motion in polycrystalline materials”. PhD thesis. Univ.-Bibliothek, 2013.DOI
- [47]Reza Darvishi Kamachali, Se-Jong Kim, and Ingo Steinbach. “Texture evolution in deformed AZ31 magnesium sheets: Experiments and phase-field study”. In: Computational Materials Science 104 (2015), pp. 193–199.DOI
- [48]Christian Schwarze et al. “Computationally efficient phase-field simulation studies using RVE sampling and statistical analysis”. In: Computational Materials Science 147 (2018), pp. 204–216.
- [49]Reza Darvishi Kamachali et al. “Numerical benchmark of phase-field simulations with elastic strains: precipitation in the presence of chemo-mechanical coupling”. In: Computational Materials Science 155 (2018), pp. 541–553.DOI
- [50]Marvin Tegeler et al. “Parallel multiphase field simulations with OpenPhase”. In: Computer Physics Communications 215 (2017), pp. 173–187.
- [51]C. Wang et al. “Machine learning in additive manufacturing: State-of-the-art and perspectives”. In: Additive Manufacturing 36 (2020), p. 101538. ISSN: 2214-8604. DOI: https://doi.org/10.1016/j.addma.2020.101538.
- [52]Mohammad Parsazadeh, Shashank Sharma, and Narendra Dahotre. “Towards the next generation of machine learning models in additive manufacturing: A review of process dependent material evolution”. In: Progress in Materials Science 135 (2023), p. 101102. ISSN: 0079-6425. DOI: https://doi.org/10.1016/j.pmatsci.2023.101102.
- [53]Ling Qiao, Yong Liu, and Jingchuan Zhu. “A focused review on machine learning aided high-throughput methods in high entropy alloy”. In: Journal of Alloys and Compounds 877 (2021), p. 160295. ISSN: 0925-8388. DOI: https://doi.org/10.1016/j.jallcom.2021.160295. URL: https://www.sciencedirect.com/science/article/pii/S0925838821017047.
- [54]Ziyuan Rao et al. “Machine learning–enabled high-entropy alloy discovery”. In: Science 378.6615 (2022), pp. 78–85. DOI: 10.1126/science.abo4940.DOI
- [55]Daniele Secci, Vanessa A. Godoy, and J. Jaime Gómez-Hernández. “Physics-Informed Neural Networks for solving transient unconfined groundwater flow”. In: Computers & Geosciences 182 (2024), p. 105494. ISSN: 0098-3004. DOI: https://doi.org/10.1016/j.cageo.2023.105494.
- [56]D. Montes de Oca Zapiain, J.A. Stewart, and R. Dingreville. “Accelerating phase-field-based microstructure evolution predictions via surrogate models trained by machine learning methods”. In: npj Comput Mater 7 (2021), p. 3. DOI: 10.1038/s41524-020-00471-8.
- [57]C. Hu, S. Martin, and R. Dingreville. “Accelerating phase-field predictions via recurrent neural networks learning the microstructure evolution in latent space”. In: Computer Methods in Applied Mechanics and Engineering 397 (2022), p. 115128. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2022.115128.
- [58]V. Oommen et al. “Learning two-phase microstructure evolution using neural operators and autoencoder architectures”. In: npj Comput Mater 8 (2022), p. 190. DOI: 10.1038/s41524-022-00876-7.DOI
- [59]Seifallah Fetni et al. “Capabilities of Auto-encoders and Principal Component Analysis of the reduction of microstructural images; Application on the acceleration of Phase-Field simulations”. In: Computational Materials Science 216 (2023), p. 111820. DOI: 10.1016/j.commatsci.2022.111820.DOI
- [60]Zhaolin Chen, Siu-Kai Lai, and Zhichun Yang. “AT-PINN: Advanced time-marching physics-informed neural network for structural vibration analysis”. In: Thin-Walled Structures 196 (2024), p. 111423. ISSN: 0263-8231. DOI: https://doi.org/10.1016/j.tws.2023.111423. URL: https://www.sciencedirect.com/science/article/pii/S0263823123009011.
- [61]Sifan Wang, Shyam Sankaran, and Paris Perdikaris. “Respecting causality for training physics-informed neural networks”. In: Computer Methods in Applied Mechanics and Engineering 421 (2024), p. 116813. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2024.116813. URL: https://www.sciencedirect.com/science/article/pii/S0045782524000690.
- [62]Khemraj Shukla, Ameya D. Jagtap, and George Em Karniadakis. “Parallel physics-informed neural networks via domain decomposition”. In: Journal of Computational Physics 447 (2021), p. 110683. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2021.110683.
- [63]C. Hüter et al. “A pragmatic dataset augmentation approach for transformation temperature prediction in steels”. In: Computational Materials Science 176 (2020), p. 109488. ISSN: 0927-0256. DOI: https://doi.org/10.1016/j.commatsci.2019.109488.
- [64]Linda L Carpenter, Sarah Yasmin, and Lawrence H Price. “A double-blind, placebo-controlled study of antidepressant augmentation with mirtazapine”. In: Biological Psychiatry 51.2 (2002), pp. 183–188. ISSN: 0006-3223. DOI: https://doi.org/10.1016/S0006-3223(01)01262-8. URL: https://www.sciencedirect.com/science/article/pii/S0006322301012628.
- [65]Yibao Li and Junseok Kim. “Phase-field simulations of crystal growth with adaptive mesh refinement”. In: International Journal of Heat and Mass Transfer 55.25 (2012), pp. 7926–7932. ISSN: 0017-9310. DOI: https://doi.org/10.1016/j.ijheatmasstransfer.2012.08.009. URL: https://www.sciencedirect.com/science/article/pii/S0017931012006308.
- [66]Abhinav Gupta et al. “An adaptive mesh refinement algorithm for phase-field fracture models: Application to brittle, cohesive, and dynamic fracture”. In: Computer Methods in Applied Mechanics and Engineering 399 (2022), p. 115347. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2022.115347. URL: https://www.sciencedirect.com/science/article/pii/S0045782522004339.
- [67]Wenqiang Xu et al. “Multi-level adaptive mesh refinement technique for phase-field method”. In: Engineering Fracture Mechanics 276 (2022), p. 108891. ISSN: 0013-7944. DOI: https://doi.org/10.1016/j.engfracmech.2022.108891. URL: https://www.sciencedirect.com/science/article/pii/S0013794422006099.
- [68]F. Freddi and L. Mingazzi. “Adaptive mesh refinement for the phase field method: A FEniCS implementation”. In: Applications in Engineering Science 14 (2023), p. 100127. ISSN: 2666-4968. DOI: https://doi.org/10.1016/j.apples.2023.100127. URL: https://www.sciencedirect.com/science/article/pii/S266649682300002X.
- [69]Ananya Bijaya et al. “A Multilevel Adaptive Mesh Scheme for Efficient Simulation of Thermomechanical Phase-Field Fracture”. In: Journal of Engineering Mechanics 150.6 (2023). DOI: 10.1061/JENMDT.EMENG-7480.DOI
- [70]Chenxi Wu et al. “A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks”. In: Computer Methods in Applied Mechanics and Engineering 403 (2023), p. 115671. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2022.115671. URL: https://www.sciencedirect.com/science/article/pii/S0045782522006260.
- [71]Ameya D. Jagtap, Kenji Kawaguchi, and George Em Karniadakis. “Adaptive activation functions accelerate convergence in deep and physics-informed neural networks”. In: Journal of Computational Physics 404 (2020), p. 109136. ISSN: 0021-9991. DOI: https://doi.org/10.1016/j.jcp.2019.109136.
- [72]Shirong Li and Xinlong Feng. “Dynamic Weight Strategy of Physics-Informed Neural Networks for the 2D Navier–Stokes Equations”. In: Entropy 24.9 (2022), p. 1254. DOI: 10.3390/e24091254.DOI
- [73]Arka Daw et al. “Mitigating propagation failures in physics-informed neural networks using retain-resample-release (r3) sampling”. In: ICML’23: Proceedings of the 40th International Conference on Machine Learning. July 2023, pp. 7264–7302.arxiv.org/abs/2207.02338
- [74]Lei Wang and Reza Darvishi Kamachali. “Incorporating elasticity into CALPHAD-informed density-based grain boundary phase diagrams reveals segregation transition in Al-Cu and Al-Cu-Mg alloys”. In: Computational Materials Science 199 (2021), p. 110717.DOI
- [75]Xuyang Zhou et al. “Revealing in-plane grain boundary composition features through machine learning from atom probe tomography data”. In: Acta materialia 226 (2022), p. 117633.arxiv.org/abs/2106.10027
- [76]Francisco Guillén-González and Giordano Tierra. “Second order schemes and time-step adaptivity for Allen–Cahn and Cahn–Hilliard models”. In: Computers & Mathematics with Applications 68.8 (2014), pp. 821–846. ISSN: 0898-1221. DOI: https://doi.org/10.1016/j.camwa.2014.07.014.
- [77]Jingying Wang et al. “A high order accurate numerical algorithm for the space-fractional Swift-Hohenberg equation”. In: Computers & Mathematics with Applications 139 (2023), pp. 216–223. ISSN: 0898-1221. DOI: https://doi.org/10.1016/j.camwa.2022.09.014.
- [78]Pi-Yueh Chuang and Lorena A. Barba. Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration. 2022. arXiv: 2205.14249 [physics.flu-dyn].arxiv.org/abs/2205.14249