betaFeedback

WorksComputer Science

Homogenization with Guaranteed Bounds via Primal-Dual Physically Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) have shown promise in solving partial differential equations (PDEs) relevant to multiscale modeling, but they often fail when applied to materials with discontinuous coefficients, such as media with piecewise constant properties. This paper introduces a dual formulation for the PINN framework to improve the reliability of the homogenization of periodic thermo-conductive composites, for both strong and variational (weak) formulations. The dual approach facilitates the derivation of guaranteed upper and lower error bounds, enabling more robust detection of PINN failure. We compare standard PINNs applied to smoothed material approximations with variational PINNs (VPINNs) using both spectral and neural network-based test functions. Our results indicate that while strong-form PINNs may outperform VPINNs in controlled settings, they are sensitive to material discontinuities and may fail without clear diagnostics. In contrast, VPINNs accommodate piecewise constant material parameters directly but require careful selection of test functions to avoid instability. Dual formulation serves as a reliable indicator of convergence quality, and its integration into PINN frameworks enhances their applicability to homogenization problems in micromechanics.

Introduction

Accurately modelling the effective behaviour of materials with complex microstructures is crucial in various scientific and engineering domains. Simulation of components made of such materials, while resolving all microstructural details, makes multi-scale modeling prohibitively computationally demanding for practical everyday use. Homogenization aims to replace a heterogeneous material with a homogeneous one that exhibits an equivalent macroscopic behaviour [8]. For most computational homogenization problems, physics-informed neural networks (PINNs) [33] currently cannot match the computational efficiency and reliability of classical Finite Element Method (FEM) and Fast Fourier Transform (FFT) approaches [12, 6], despite promising theoretical advantages [14]. However, PINNs demonstrate clear advantages for mesh-free computation of complex microstructures, multi-query parametric studies, and scenarios involving irregular geometries where traditional meshing becomes problematic [35].

Recent advances in PINN architectures have begun addressing fundamental limitations in multiscale problems [32, 24, 39]. These developments, combined with the inherent advantages of mesh-free computation and rapid evaluation for trained networks, position PINNs as valuable tools for specific applications while classical methods remain optimal for computational homogenization workflows in production.

A critical gap remains in establishing rigorous error bounds and robust formulations for elliptic partial differential equations (PDEs) within the PINN framework. While existing PINN approaches primarily rely on the strong form of the PDE residual, this formulation can be sensitive to solution regularity and may not provide reliable error estimates [21, 37, 29]. Variational PINNs (VPINNs) address some of these limitations by incorporating weak formulations that utilize test functions and integrate the PDE residual over the domain, thereby imposing weaker regularity requirements and potentially enhancing solution stability. However, VPINNs remain sensitive to the choice of test functions and still lack robustness in error estimation [7, 20].

To address these challenges in PINN-based homogenization, we propose a scheme that enables the calculation of guaranteed upper and lower bounds for effective parameters. Using two microstructure examples, we demonstrate how classic PINNs can fail when applied to piecewise constant materials. Although sophisticated techniques exist for handling discontinuous materials, including GFEM [22], X-FEM [1], the immersed boundary method [25], domain decomposition PINNs [17], and various adaptive PINN approaches [26, 43], robust application of advanced PINNs in homogenization settings remains under development.

Our approach employs both primal and dual formulations of the elliptic PDE governing the homogenization problem, rather than relying solely on the primal formulation. The dual formulation provides complementary information that enables more accurate and reliable bounds on effective material properties, as demonstrated for classical homogenization methods by Gaynutdinova et al. (2022, 2023). For the network architecture, we adopt the frameworks of Jiang et al. (2023) and Soyarslan and Pradas [39], which incorporate periodic boundary conditions directly without loss penalty terms. We extend their architecture to solve the dual formulation problem while satisfying the divergence-free constraint [34, 9], then use the primal and dual solutions to derive error bounds. Recognizing that the strong form can still yield poor performance in classic PINNs, we also implement the primal-dual approach for VPINNs and compare results across different test function choices [2, 36]. Our results reveal that strong-form PINNs outperform both VPINNs and FEM benchmarks only when materials exhibit sufficiently gradual phase transitions, while VPINNs show sensitivity to test function selection. In all cases, however, the guaranteed bounds to the effective parameters can be utilized to assess the quality of the computed results.

The rest of this work is structured as follows. In § 2, we recall the zeroth-order homogenization theory for both primal and dual formulations in the steady-state linear 2D heat equation. In § 3, we introduce the PINN architecture and construct the strong training loss for the dual formulation, as well as the variational loss for both primal and dual formulations. In Subsection 4.1.1, we compare the performance of the classic PINNs for different material functions and address a common mode of PINN failure, whereas in Subsection 4.1.2 we compare the performance of VPINNs for different types and cardinalities of test bases; in both cases, for a single inclusion. Subsection 4.2 then extends this discussion to a microstructure with multiple inclusions. § 5 summarizes our findings and discusses the pros and cons of each method, and § 6 provides the outlook for future applications.

Primal and Dual Formulation of Homogenization

In this part, we employ computational homogenization, which determines the effective properties of heterogeneous materials by solving boundary value problems on their representative periodic cells. First, we introduce the governing equation for the homogenization of linear steady-state heat transport in 2D by recalling the strong form of the equation, from which the typical PINN loss is constructed [32, 24, 39] (Jiang et al. 2023; Wu et al. 2023), and introduce its dual form. Then we present the variational forms for both the primal and dual formulations, which are used in most classical numerical methods, and will serve as the basis for the VPINN approach. Finally, we show how to calculate the estimates of the homogenized conductivity matrix, as well as its guaranteed upper and lower bounds.

Strong form

Let us consider a periodic unit cell X⊂R2X \subset\mathbb{R}^{2} representing a periodic heterogeneous solid, where its microstructural thermal conductivity second-order tensor can be represented in the matrix notation as a 2×22 \times2 symmetric matrix A\mathbf{A}. Its spatial variability within the periodic cell is then expressed as a function of the spatial coordinate x\mathbf{x} as A(x):X→Rsym2×2\mathbf{A}(\mathbf{x}): X \to\mathbb{R}_{\mathrm{sym}}^{2 \times2}, which is uniformly positive definite, with each component essentially bounded in XX. According to the zeroth-order homogenization theory [27, 19], the temperature field u(x)u(\mathbf{x}) within the unit cell takes the form

u(x)=ξTx+u~(x),x∈X,(1)u(\mathbf{x}) = \boldsymbol{\xi}^{\mathsf{T}}\mathbf{x} + \widetilde{u}(\mathbf{x}), \quad\mathbf{x} \in X, \tag*{(1)}

where ξTx\boldsymbol{\xi}^{\mathsf{T}}\mathbf{x} represents the averaged contribution for the prescribed macroscopic temperature gradient ξ\boldsymbol{\xi} and u~(x)\widetilde{u}(\mathbf{x}) is the micro-fluctuation part caused by the material heterogeneity. The temperature fluctuation u~(x)\widetilde{u}(\mathbf{x}) is a XX-periodic function in the x1−x2x_{1}-x_{2} plane, x=[x1,x2]\mathbf{x} = [x_{1}, x_{2}].

Under steady-state conditions with no heat sources, the heat flux j(x)\boldsymbol{j}(x) is related to the temperature gradient e(x)=∇u(x)=[∂u∂x1,∂u∂x2]T\boldsymbol{e}(x)=\nabla u(x)=\left[\frac{\partial u}{\partial x_1},\frac{\partial u}{\partial x_2}\right]^{\mathrm{T}} by Fourier’s law as:

j(x)=−A(x)e(x)=−A(x)∇u(x),(2)\boldsymbol{j}(x)=-\boldsymbol{A}(x)\boldsymbol{e}(x)=-\boldsymbol{A}(x)\nabla u(x), \tag*{(2)}

and the divergence of the heat flux must be zero, i.e.,

∇⋅j(x)=0.(3)\nabla\cdot\boldsymbol{j}(x)=0. \tag*{(3)}

The strong form of the governing differential equation of the steady-state heat conduction then requires one to find u~∈Cper2(X)\widetilde{u}\in C_{\mathrm{per}}^{2}(X) such that:

∇⋅[A(x)(ξ+∇u~(x))]=0, ∀x∈X,(4)\nabla\cdot\left[\boldsymbol{A}(x)\left(\boldsymbol{\xi}+\nabla\widetilde{u}(x)\right)\right]=0,\ \forall x\in X, \tag*{(4)}

where ∇⋅\nabla\cdot denotes the divergence operator, i.e., ∇⋅f=∂f1∂x1+∂f2∂x2\nabla\cdot\boldsymbol{f}=\frac{\partial f_1}{\partial x_1}+\frac{\partial f_2}{\partial x_2} and Cper2(X)C_{\mathrm{per}}^{2}(X) denotes the space of twice-differentiable periodic continuous function on XX.

The dual form of Eq. (4) is defined for the thermal resistivity A−1(x)\boldsymbol{A}^{-1}(x) ([40]. For the heat transfer problem, we can derive it analogously from the property of the temperature gradient field e\boldsymbol{e} ([4, 28], following the Helmholtz decomposition theorem ([19], i.e.:

∇×e(x)=0, ∀x∈X.(5)\nabla\times\boldsymbol{e}(x)=0,\ \forall x\in X. \tag*{(5)}

The vector field e(x)\boldsymbol{e}(x) then relates back to the heat flux through the resistivity tensor:

e(x)=−A−1(x)j(x).(6)\boldsymbol{e}(x)=-\boldsymbol{A}^{-1}(x)\boldsymbol{j}(x). \tag*{(6)}

We can again decompose the heat flux j(x)\boldsymbol{j}(x) into a constant part ζ\boldsymbol{\zeta} and a fluctuating part v~(x)\widetilde{\boldsymbol{v}}(x), resulting in the dual form of (4) as

∇×[A−1(x)(ζ+v~(x))]=0, ∀x∈X.(7)\nabla\times\left[\boldsymbol{A}^{-1}(x)\left(\boldsymbol{\zeta}+\widetilde{\boldsymbol{v}}(x)\right)\right]=0,\ \forall x\in X. \tag*{(7)}

Importantly, this new vector field v~(x)\widetilde{\boldsymbol{v}}(x) has to satisfy the divergence-free condition of the admissible heat flux fields, i.e.:

∇⋅v~(x)=0.(8)\nabla\cdot\widetilde{\boldsymbol{v}}(x)=0. \tag*{(8)}

In the two-dimensional setting, every divergence-free vector function v~(x)\widetilde{\boldsymbol{v}}(x) can be derived from some periodic scalar function w~(x)\widetilde{w}(x) (the so-called stream function) as

v~(x)=Q∇w~(x),(9)\widetilde{\boldsymbol{v}}(x)=Q\nabla\widetilde{w}(x), \tag*{(9)}

where QQ is a 90∘90^\circ rotation matrix in the x1x_1–x2x_2 plane ([13, 28]) of the form

Q=[0−110].(10)Q=\begin{bmatrix}0&-1\\1&0\end{bmatrix}. \tag*{(10)}

The divergence of vector v~(x)\tilde{\mathbf{v}}(x) can then be written as

∇⋅v~(x)=[∂∂x1,∂∂x2]⋅[∂w~∂x2−∂w~∂x1]=−∂2w~∂x2∂x1+∂2w~∂x1∂x2,(11)\nabla\cdot\tilde{\mathbf{v}}(x)=\left[\frac{\partial}{\partial x_1},\frac{\partial}{\partial x_2}\right]\cdot\left[\begin{array}{c}\frac{\partial\tilde{w}}{\partial x_2}\\-\frac{\partial\tilde{w}}{\partial x_1}\end{array}\right]=-\frac{\partial^2\tilde{w}}{\partial x_2\partial x_1}+\frac{\partial^2\tilde{w}}{\partial x_1\partial x_2}, \tag*{(11)}

which is always equal to zero, thanks to the symmetry of partial derivatives. Then, the problem (7) can be written as to find w~(x)∈Cper2(X)\tilde{w}(x)\in C_{\mathrm{per}}^2(X) such that

∇×[A−1(x)(ζ+Q∇w~(x))]=0, ∀x∈X,(12)\nabla\times\left[A^{-1}(x)(\zeta+Q\nabla\tilde{w}(x))\right]=0,\ \forall x\in X, \tag*{(12)}

with the divergence-free constraint (8) is automatically fulfilled. Note that in order to apply the curl operator ∇×\nabla\times to a 2D vector function f\boldsymbol{f}, the function has to be expanded to a 3D vector function [f1,f2,f3=0][f_1,f_2,f_3=0]. The condition ∇×f=0\nabla\times\boldsymbol{f}=0 can then be written as

[∂f3∂x2−∂f2∂x3,∂f1∂x3−∂f3∂x1,∂f2∂x1−∂f1∂x2]=0.(13)\left[\frac{\partial f_3}{\partial x_2}-\frac{\partial f_2}{\partial x_3},\frac{\partial f_1}{\partial x_3}-\frac{\partial f_3}{\partial x_1},\frac{\partial f_2}{\partial x_1}-\frac{\partial f_1}{\partial x_2}\right]=0. \tag*{(13)}

Since f3=0f_3=0 and f\boldsymbol{f} is a function in the x1x_1–x2x_2 plane, the first two components are always zero, and the equation (12) simplifies to contain only the last component, thus reducing to a scalar problem similar to Eq. (4). We keep the original notation for brevity, but from now on, when we write ∇×f\nabla\times\boldsymbol{f}, we actually mean its scalar form ∂f2∂x1−∂f1∂x2\frac{\partial f_2}{\partial x_1}-\frac{\partial f_1}{\partial x_2}.

Weak form

Despite promising theoretical advantages, strong-form PINNs currently cannot match the computational efficiency and reliability of classical computational approaches for most PDEs ([14]). As the analytical solution of problems (4) and (12) is usually unavailable, most frameworks, such as the finite element method (FEM) ([42]) and Fourier-Galerkin method ([30, 38]), rely on their respective variational (weak) forms. This allows for the relaxation of the solution regularity requirement from Cper2(X)C_{\mathrm{per}}^2(X) to space Hper1(X)={u∈L2(X,R);uper∈Hloc1(R2,R)}H_{\mathrm{per}}^1(X)=\{u\in L^2(X,\mathbb{R});u_{\mathrm{per}}\in H_{\mathrm{loc}}^1(\mathbb{R}^2,\mathbb{R})\}, where uperu_{\mathrm{per}} is the XX-periodic extension of uu and Hloc1(R2,R)H_{\mathrm{loc}}^1(\mathbb{R}^2,\mathbb{R}) are functions with squared gradients integrable on every compact subset of R2\mathbb{R}^2. Instead of solving (4), we want to find u~∈Hper1(X)\tilde{u}\in H_{\mathrm{per}}^1(X) such that

∫X(∇ϕ(x))TA(x)(ξ+∇u~(x))dx=0, for all ϕ∈Hper1(X),(14)\int_X(\nabla\phi(x))^{\mathrm{T}}A(x)(\xi+\nabla\tilde{u}(x))\mathrm{d}x=0,\ \text{for all }\phi\in H_{\mathrm{per}}^1(X), \tag*{(14)}

where ϕ(x)\phi(x) is a test function. The boundary term ∫∂X(n⋅∇ϕ)u~(x)ds=0\int_{\partial X}(n\cdot\nabla\phi)\tilde{u}(x)\mathrm{d}s=0, which appears alongside (14) after the integration by parts, vanishes automatically, since it has opposite values on the opposite boundaries of XX due to periodicity. Similarly, the variational form of (12) is to find w~∈Hper1(X)\tilde{w}\in H_{\mathrm{per}}^1(X) such that

∫X(Q∇ψ(x))TA−1(x)(ξ+Q∇w~(x))dx=0, for all ψ∈Hper1(X),(15)\int_X(Q\nabla\psi(x))^{\mathrm{T}}A^{-1}(x)(\xi+Q\nabla\tilde{w}(x))\mathrm{d}x=0,\ \text{for all }\psi\in H_{\mathrm{per}}^1(X), \tag*{(15)}

with a test function ψ(x)∈Hper1(X)\psi(x)\in H_{\mathrm{per}}^1(X).

Upper and lower bound to homogenized parameters

The exact solution of (14) is also the minimizer of the related homogenization problem [19], which reads to find a homogenized coefficient matrix A∗∈R2×2\mathbf{A}^{*} \in\mathbb{R}^{2 \times2} such that

⟨A∗ξ,ξ⟩R2=inf⁡u~∈Hper1(X)1∣X∣∫X⟨A(x)(ξ+∇u~(x)),ξ+∇u~(x)⟩R2 dx, ∀ξ∈R2,(16)\langle\mathbf{A}^{*}\boldsymbol{\xi},\boldsymbol{\xi}\rangle_{\mathbb{R}^{2}}=\inf_{\widetilde{u}\in H^{1}_{\mathrm{per}}(X)}\frac{1}{|X|}\int_{X}\langle\mathbf{A}(\mathbf{x})(\boldsymbol{\xi}+\nabla\widetilde{u}(\mathbf{x})),\boldsymbol{\xi}+\nabla\widetilde{u}(\mathbf{x})\rangle_{\mathbb{R}^{2}}\,\mathrm{d}\mathbf{x},\ \forall\boldsymbol{\xi}\in\mathbb{R}^{2}, \tag*{(16)}

where ∣X∣|X| denotes the area of the domain XX, and ⟨u,v⟩R2\langle\mathbf{u},\mathbf{v}\rangle_{\mathbb{R}^{2}} denotes the inner product of vectors u\mathbf{u} and v\mathbf{v} of the Euclidean space R2\mathbb{R}^{2}.

Conversely, the inverse B∗\mathbf{B}^{*} of the homogenized coefficient A∗\mathbf{A}^{*} can be calculated using the dual formulation [15, 40] (Gaynutdinova et al. 2022):

⟨B∗ζ,ζ⟩R2=inf⁡w~∈Hper1(X)1∣X∣∫X⟨A−1(x)(ζ+Q∇w~(x)),ζ+Q∇w~(x)⟩R2 dx, ∀ζ∈R2.(17)\langle\mathbf{B}^{*}\boldsymbol{\zeta},\boldsymbol{\zeta}\rangle_{\mathbb{R}^{2}}=\inf_{\widetilde{w}\in H^{1}_{\mathrm{per}}(X)}\frac{1}{|X|}\int_{X}\langle\mathbf{A}^{-1}(\mathbf{x})(\boldsymbol{\zeta}+\mathbf{Q}\nabla\widetilde{w}(\mathbf{x})),\boldsymbol{\zeta}+\mathbf{Q}\nabla\widetilde{w}(\mathbf{x})\rangle_{\mathbb{R}^{2}}\,\mathrm{d}\mathbf{x},\ \forall\boldsymbol{\zeta}\in\mathbb{R}^{2}. \tag*{(17)}

Given the variational structure of (16) and (17), for which the true minimizer exists in the infinite-dimensional space Hper1(X)H^{1}_{\mathrm{per}}(X), substituting any minimizer over subspaces of Hper1(X)H^{1}_{\mathrm{per}}(X) (i.e., approximate solutions u~h\widetilde{u}_{h} and w~h\widetilde{w}_{h}) into the integrals in (16) and (17), respectively, for given ξ\boldsymbol{\xi} and ζ\boldsymbol{\zeta} yields larger values of the effective coefficients than for the true minimizers, and thus it yields the upper bounds Ah∗\mathbf{A}^{*}_{h} and Bh∗\mathbf{B}^{*}_{h} to A∗\mathbf{A}^{*} and B∗\mathbf{B}^{*} in the sense that

(Bh∗)−1⪯(B∗)−1=A∗⪯Ah∗,(18)(\mathbf{B}^{*}_{h})^{-1}\preceq(\mathbf{B}^{*})^{-1}=\mathbf{A}^{*}\preceq\mathbf{A}^{*}_{h}, \tag*{(18)}

where the partial ordering K⪯L\mathbf{K}\preceq\mathbf{L} means that vTKv≤vTLv\mathbf{v}^{\mathsf{T}}\mathbf{K}\mathbf{v}\le\mathbf{v}^{\mathsf{T}}\mathbf{L}\mathbf{v} for all v∈R2\mathbf{v}\in\mathbb{R}^{2}.

In practice, we solve (4) or (14) (as well as (12) and (15)) for given ξ\boldsymbol{\xi} and ζ∈{[1,0]T,[0,1]T}\boldsymbol{\zeta}\in\{[1,0]^{\mathsf{T}},[0,1]^{\mathsf{T}}\} to compute the diagonal elements of Ah∗\mathbf{A}^{*}_{h} and Bh∗\mathbf{B}^{*}_{h}.

Physically Informed Neural Networks for Homogenization

To show how existing PINN homogenization approaches can be improved by introducing the dual form training loss, we adopt the PINN architecture introduced by Jiang et al. (2023) (Figure 1), which was demonstrated to produce a good approximation of the solution of (4) for a wide range of different material functions with smooth coefficients, see Figure 1. This network is characterized by a unique periodic layer that employs learnable cosine functions. This design choice ensures that the network’s output is inherently periodic, thereby eliminating the need to incorporate a boundary condition term into the training loss.

Scheme of the primal PINN architecture

Figure 1. Scheme of the primal PINN architecture introduced by Jiang et al. [18]. The network inputs the microscopic coordinates [x1,x2][x_1, x_2] and outputs the fluctuating temperature u~NN\widetilde{u}_{\mathrm{NN}}. Identical architecture is used for the dual formulation, where the output is the stream function w~NN\widetilde{w}_{\mathrm{NN}}.

The network takes as input the microscopic coordinates x1x_{1} and x2x_{2} and produces output in one of two forms: it can either deliver the fluctuating temperature u~NN\widetilde{u}_{\mathrm{NN}} in the primal form or provide the stream function w~NN\widetilde{w}_{\mathrm{NN}} in the dual form. Thus, by training two separate networks with the same architecture, we can obtain a primal and a dual solution to the homogenization problem. The architecture of each network includes the periodic layer alongside one or more sequential residual layers, which utilize smooth activation functions like the hyperbolic tangent.

Strong form

The PDE residual from (4) or (12) is used as the PINN training loss, i.e.:

Lp,s:=1∣X∣∫X(∇⋅[A(x)(ξ+∇u~NN(x))])2 dx(19)\mathcal{L}_{\mathrm{p,s}} := \frac{1}{\lvert X\rvert}\int_X \left(\nabla\cdot\left[\mathbf{A}(x)\left(\boldsymbol{\xi}+\nabla\widetilde{u}_{\mathrm{NN}}(x)\right)\right]\right)^2 \,\mathrm{d}x \tag*{(19)}

for the primal strong-form PINN, and

Ld,s:=1∣X∣∫X(∇×[A−1(x)(ζ+Q∇w~NN(x))])2 dx(20)\mathcal{L}_{\mathrm{d,s}} := \frac{1}{\lvert X\rvert}\int_X \left(\nabla\times\left[\mathbf{A}^{-1}(x)\left(\boldsymbol{\zeta}+\mathbf{Q}\nabla\widetilde{w}_{\mathrm{NN}}(x)\right)\right]\right)^2 \,\mathrm{d}x \tag*{(20)}

for the dual strong-form PINN.

Weak form

In this paper, we also aim to address certain limitations of PINNs for the homogenization of piecewise constant materials that are often encountered in the laminate or matrix-inclusion forms. Because the strong formulation is used in training the networks, the material function A(x)\mathbf{A}(x) is also assumed to have continuous first-order gradients. While the material smoothness condition is not strictly necessary for the technical implementation of PINNs, ignoring it often leads to poor results [16], so an artificial smoothing of the material function is usually employed. Moreover, in our particular homogenization setting, naively implementing piecewise-constant material coefficients leads to failure, as the network converges to a near-constant solution because the term ∇⋅(A(x)ξ)\nabla\cdot(\mathbf{A}(x)\boldsymbol{\xi}) is zero in all possible collocation points. Introducing material smoothing, however, alters the underlying problem and introduces a material approximation error into the estimates of the homogenized parameter. We later show that even for smooth functions, an insufficiently gradual transition between the material phases can cause poor PINN solutions, because ∇⋅(A(x)ξ)\nabla\cdot(\mathbf{A}(x)\boldsymbol{\xi}) becomes very large at the interface between the materials and very small elsewhere.

A potential remedy lies in replacing the strong form of the governing PDE with its weak form, which allows for the relaxation of the smoothness constraint in both the material function and the solution. While there are many sophisticated methods for addressing material discontinuities [26, 23], we aim to test the limits of the standard approach by letting the VPINN have an identical architecture to the PINN described above. It is important to note that the weak formulation also broadens the options for activation functions; however, to isolate the impact of the formulation itself, we will continue to use the hyperbolic tangent activation function.

We implement the loss function for Robust VPINNs introduced by Rojas et al. [36]. For the primal neural network solution u~NN\widetilde{u}_{\mathrm{NN}} and a basis of test functions ϕn(x)∈Hper1(X)\phi_n(x) \in H_{\mathrm{per}}^1(X), n=1,…,Ntn=1,\ldots,N_t we compute the vector of residuals rp(u~NN)=[rp1,…,rpNt]T\boldsymbol{r}_{\mathrm{p}}(\widetilde{u}_{\mathrm{NN}})=[r_{\mathrm{p}}^1,\ldots,r_{\mathrm{p}}^{N_t}]^{\mathrm{T}}:

rpn(u~NN)=∫X(∇ϕn(x))TA(x)(ξ+∇u~NN(x)) dx,n=1,…,Nt.(21)r_{\mathrm{p}}^n(\widetilde{u}_{\mathrm{NN}})=\int_X(\nabla\phi_n(x))^{\mathrm{T}}A(x)(\xi+\nabla\widetilde{u}_{\mathrm{NN}}(x))\,\mathrm{d}x,\quad n=1,\ldots,N_t. \tag*{(21)}

As shown in [36], minimizing the sum of squared residuals with respect to normalized individual test functions is equivalent to minimizing the functional

Lp,w:=rpT(u~NN)G−1rp(u~NN),(22)\mathcal{L}_{\mathrm{p,w}}:=\boldsymbol{r}_{\mathrm{p}}^{\mathrm{T}}(\widetilde{u}_{\mathrm{NN}})G^{-1}\boldsymbol{r}_{\mathrm{p}}(\widetilde{u}_{\mathrm{NN}}), \tag*{(22)}

where GG is a symmetric positive definite Gram matrix, Gnm=∫X(∇ϕn)T∇ϕm dxG_{nm}=\int_X(\nabla\phi_n)^{\mathrm{T}}\nabla\phi_m\,\mathrm{d}x. The dual weak loss functional is defined analogously with the test functions ψn(x)∈Hper1(X)\psi_n(x)\in H_{\mathrm{per}}^1(X), n=1,…,Ntn=1,\ldots,N_t, where the weak dual residual is

rdn(w~NN)=∫X(Q∇ψn(x))TA−1(x)(ζ+Q∇w~NN(x)) dx,n=1,…,Nt.(23)r_{\mathrm{d}}^n(\widetilde{w}_{\mathrm{NN}})=\int_X(Q\nabla\psi_n(x))^{\mathrm{T}}A^{-1}(x)(\zeta+Q\nabla\widetilde{w}_{\mathrm{NN}}(x))\,\mathrm{d}x,\quad n=1,\ldots,N_t. \tag*{(23)}

Numerical Experiments

This section demonstrates how training PINNs in the dual formulation of the problem can complement the solution in the primal form, and compares the effectiveness of VPINNs against classic PINNs for unit cells with a singular (§ 4.1) and multiple (§ 4.2) inclusions.

Singular Inclusion

Consider a square domain XX of the size 2π×2π2\pi\times2\pi with a centrally placed square inclusion of the size π×π\pi\times\pi with isotropic material phases, shown in Figure 2. For this case, the effective conductivity can be computed exactly [31] as

Smooth approximations of the material distribution for three values of $\varepsilon$

Figure 2. Smooth approximations As,ε(x)A_{s,\varepsilon}(\boldsymbol{x}) of the material distribution depending on the parameter ε\varepsilon.

A∗=γeffI=γmatγmat+3γinc3γmat+γinc I,(24)A^{*}=\gamma^{\mathrm{eff}}I=\gamma_{\mathrm{mat}}\sqrt{\frac{\gamma_{\mathrm{mat}}+3\gamma_{\mathrm{inc}}}{3\gamma_{\mathrm{mat}}+\gamma_{\mathrm{inc}}}}\,I, \tag*{(24)}

where γmat\gamma_{\mathrm{mat}} and γinc\gamma_{\mathrm{inc}} are thermal conductivities of the matrix and inclusion, respectively. With γmat=1\gamma_{\mathrm{mat}}=1 and γinc=0.1\gamma_{\mathrm{inc}}=0.1, formula (24) yields $\gamma^{\mathrm{eff}}\approx 0.6476.

Together with the analytical exact homogenized conductivity γeff\gamma^{\mathrm{eff}}, we use a FEM solution for the benchmark on u~\tilde{u} and w~\tilde{w} (cf. Figure 3). The primal and dual FEM solutions provide guaranteed upper and lower bounds to γeff\gamma^{\mathrm{eff}} [10]. For the sake of simplicity, we use a regular grid with triangular elements and continuous piecewise linear functions. Since the material distribution is symmetric, we only have to compute the solution for one macroscopic gradient (ξ\xi, ζ=[1,0]T\zeta= [1, 0]^{T}), and we will be comparing the estimates of the first diagonal element of the homogenized heat conductivity matrix only. For the solution on a regular mesh containing 128×128128 \times128 unique degrees of freedom (DoFs) (129×129129 \times129 grid with periodic boundary nodes), these bounds are very close to the exact analytical effective conductivity (within ±0.04%\pm0.04\% of relative error).

Figure 3. Benchmark primal and dual solutions for ξ=[1,0]T\boldsymbol{\xi}=[1,0]^{\mathsf{T}}, obtained with FEM, using discretization into 128 ×\times 128 unique DoFs.

In all experiments, we consider four neural network architectures of varying parameter sizes, as described in Table 1 for both primal and dual formulations. To reduce the impact of the choice of the collocation points, we train all the networks using the same grid as the highest resolution Finite Element Method (FEM) solution, which has 16,384 collocation points. These points are then utilized for numerical integration with the trapezoidal rule to calculate the training losses in Eqs. (19)–(21) and (23). They are also used for effective parameter estimation in Eqs. (16) and (17).

№ neurons, periodic layer№ neurons, per residual layer№ residual layers№ parameters
44165
10102391
202031 801
5050515 601

Table 1. Configurations of the considered neural networks for both primal/dual and strong/weak formulations, single inclusion.

Strong form

As discussed earlier, problems with piecewise constant material distribution cannot be solved with the classic PINNs based on the strong formulation. As a workaround, we adopt a smooth (Cper∞(X)C^{\infty}_{\mathrm{per}}(X)) approximation of the piecewise constant material function, similarly to Jiang et al. (2023). In this work, the approximation takes the form

As,ε(x)=[1−0.9⋅pε(x1)⋅pε(x2)]I,(25)A_{s,\varepsilon}(x)=\left[1-0.9\cdot p_{\varepsilon}(x_{1})\cdot p_{\varepsilon}(x_{2})\right]I, \tag*{(25)}

where

pε(xi)=12(1+tanh⁡[1εsin⁡(xi−π2)]),(26)p_{\varepsilon}(x_i)=\frac{1}{2}\left(1+\tanh\left[\frac{1}{\varepsilon}\sin\left(x_i-\frac{\pi}{2}\right)\right]\right), \tag*{(26)}

and where parameter ε>0\varepsilon>0 controls the steepness of the material transition; smaller ε\varepsilon leads to a narrower interpolation area between the two materials, see examples in Figure 2.

To establish the link between the smoothness of the material and the solution quality, we trained all network architectures of Table 1 on material distributions with ε∈{1/10,1/20,1/30,1/40}\varepsilon\in\{1/10,1/20,1/30,1/40\} for 40 000 epochs with the Adam optimizer and the starting learning rate 10−510^{-5}. The networks output the primal and dual solutions, which can be used to compute estimates of the homogenized conductivity, as well as the guaranteed upper and lower bounds. For the estimates, we approximate the integrals in (16) and (17) by calculating the terms ∇u~(x)\nabla\tilde{u}(x) and Q∇w~(x)Q\nabla\tilde{w}(x) in the collocation points, and then using trapezoidal rule. Since these gradients must be calculated during training, the gap between the primal and dual estimates provides a fast additional convergence metric to follow during the training process. The final estimates are summarized in Figure 4 as dots connected by bold dashed lines. Here we can see that these estimates can be quite precise, i.e., have a very low relative difference (A1,1p−A1,1d)/A1,1p(A^{p}_{1,1}-A^{d}_{1,1})/A^{p}_{1,1} between them (i.e., primal-dual gap), especially for the wider material transition (ε=1/10,1/20\varepsilon=1/10,1/20). However, the order of Eq. (18) is not guaranteed for these quick estimates, due to the imprecision of the integration [15]. Furthermore, the estimates only hold for the smooth material distribution As,ε(x)A_{s,\varepsilon}(x), which introduces its own approximation error, and the networks converge to an effective conductivity around 0.5% higher than the analytical solution.

Comparison of the primal and dual estimates and guaranteed bounds of the PINN solutions

Figure 4. Comparison of the primal and dual estimates (-●-) and guaranteed bounds (-⋆-) of the PINN solutions depending on the number of network parameters and the transition parameter ϵ\epsilon of the material approximation.

In principle, the computation of the guaranteed bounds requires two conditions to be satisfied: (i) the integrals in Eqs. (16) and (17) are calculated precisely; and (ii) the approximation must satisfy the required conditions. imate solutions u~(x)\tilde{u}(x) and w~(x)\tilde{w}(x) are from the subspaces of Hper1(X)H_{\mathrm{per}}^1(X). To tackle the challenge of precisely integrating the PINN solution (since it can be difficult to derive a closed form), we project it onto finite-element functions, which also form a subspace of Hper1(X)H_{\mathrm{per}}^1(X). The simplest approach is to set the projected solution in the nodes of some finite element mesh to be equal to the PINN values at these points and interpolate linearly between them. As a result, the terms ∇u~(x)\nabla\tilde{u}(x) and Q∇w~(x)Q\nabla\tilde{w}(x) remain constant within each triangular subdomain. If the material distribution is piecewise constant and the mesh is designed to align with the material phases, both the primal and dual flux will also be constant in each triangular subdomain. This makes the exact integration of the function trivial. Here, we use the same regular mesh that was used for the FEM benchmark computations, which also aligns with the training collocation points.

The resulting bounds to the conductivity parameter are denoted by stars connected by thin lines in Figure 4. Because the chosen material approximation increases the effective conductivity, the bounds obtained from the solution with ϵ∈{1/10,1/20,1/30}\epsilon\in\{1/10, 1/20, 1/30\} are closer to each other and to the exact value. On the other hand, the lower bounds are actually below the exact value, i.e., they are guaranteed, unlike the “quick” estimates.

Generally, the primal PINN estimates converge slightly faster for an increasing number of parameters than their dual counterpart, unlike the FEM primal and dual solutions, which converge at the same rate. For smaller ε\varepsilon, larger networks are required to achieve the same primal-dual gap. The primal-dual gap decreases the fastest for the material with the most gradual transition (ε=1/10\varepsilon= 1/10), at the price of wider guaranteed bounds, when the actual discontinuous material is considered. The PINN with the lowest number of DoFs can yield an estimate on par with the FEM benchmark if the material transition is sufficiently gradual. The best result with respect to the analytical effective conductivity is achieved for the largest network of 15,601 parameters and ε=1/30\varepsilon= 1/30. On the other hand, even large networks produce poor solutions for the considered ε=1/40\varepsilon= 1/40 and smaller.

The results reveal that the greatest danger in utilizing only primal PINNs for solving piecewise constant materials (or even their smooth approximations) is the inability to identify when the achieved solution is poor. When solely looking at the training process (Figure 5–Figure 6) and the resulting residual fields (Figure 7–Figure 8), the results might not indicate poor convergence. In particular, compare training of the PINNs for ε=1/30\varepsilon= 1/30 (Figure 5) and ε=1/40\varepsilon= 1/40 (Figure 6), where the training curves look similar, but the achieved parameter estimates differ substantially. In the absence of the reference solution, it can also be hard to identify the point of failure, as the residuals are even lower in the area outside the material transition for the ε=1/40\varepsilon= 1/40 solution than for the ε=1/30\varepsilon= 1/30 solution, see Figure 7 and Figure 8. Only the difference between the primal and dual estimates reveals the poor quality of the PINN solutions.

Training loss plots for the primal and dual solutions and the primal-dual gap

Figure 5. Training loss of the primal and dual solutions and the gap between the estimates, for the PINNs with the transition parameter ε=1/30\varepsilon= 1/30.

Training loss of the primal and dual solutions and the gap between the estimates for the PINNs with the transition parameter $\varepsilon = 1/40$

Figure 6. Training loss of the primal and dual solutions and the gap between the estimates, for the PINNs with the transition parameter ε=1/40\varepsilon= 1/40.

Primal solutions and residuals of select PINNs depending on the transition parameter $\varepsilon$

Figure 7. Primal solutions (top) and the residuals (bottom) of select PINNs depending on the transition parameter ε\varepsilon.

Dual solutions and residuals of select PINNs for different transition parameters

Figure 8. Dual solutions (top) and the residuals (bottom) of select PINNs depending on the transition parameter ε\varepsilon.

Weak form

The inability of the strong-form PINNs to handle discontinuous material distribution can be remedied by considering the weak form of the underlying PDE in the Variational PINNs (VPINNs). For a direct comparison of their performance to the PINNs in numerical homogenization, we consider the same network architectures as in § 4.1.1, and we use the collocation points on the same 128×128128 \times128 grid for evaluating the integrals in Eqs. (14) and (15). As we expect the weak formulation to handle the piecewise constant material, we only train for the original, piecewise constant coefficient matrix A(x)\mathbf{A}(x). However, in VPINNs, the choice of appropriate test functions becomes crucial (Berrone et al., [2]). Here, we investigate two types of test function bases: a spectral basis (paragraph VSPINN below) and a neural network basis (paragraph VNPINN below).

VPINN with spectral test functions (VSPINN). First, we consider a typical basis for the periodic square domain Y=(0,2π)×(0,2π)Y = (0, 2\pi) \times(0, 2\pi) derived from the Fourier transform in the form:

ϕs,m,n:=sin⁡(mx1+nx2),\phi_{s,m,n} := \sin(mx_1 + nx_2),
ϕc,m,n:=cos⁡(mx1+nx2),\phi_{c,m,n} := \cos(mx_1 + nx_2),

where m=0,1,…Mm = 0, 1, \ldots M, n=0,1,…Nn = 0, 1, \ldots N, with (m,n)≠(0,0)(m,n) \ne(0,0). The Gram matrix G\mathbf{G} of Eq. (22) is then diagonal and thus easily invertible. Each diagonal element of the inverse Gram matrix corresponding to the sine or cosine test function with parameters mm and nn can be calculated as

diag⁡(Gs,m,n−1)=diag⁡(Gc,m,n−1)=1m2+n2.(27)\operatorname{diag}(G^{-1}_{s,m,n}) = \operatorname{diag}(G^{-1}_{c,m,n}) = \frac{1}{m^2+n^2}. \tag*{(27)}

The primal and dual solutions were trained with a basis of size Nt=70N_t = 70 spectral test functions (M=N=5M = N = 5) and Nt=126N_t = 126 functions (M=N=7M = N = 7) with the architecture setups from Table 1. The resulting estimates and bounds are shown in Figure 9.

Comparison of the primal and dual estimates and guaranteed bounds of the VSPINN solutions

Figure 9. Comparison of the primal and dual estimates (-●-) and guaranteed bounds (-⋆-) of the VSPINN solutions depending on the number of network parameters and the number of spectral test functions.

Here we can see that the variational PINNs with the spectral test functions (VSPINNs) can handle the discontinuity in the material and yield more precise estimates than PINNs with the small transition phase ε=1/40\varepsilon= 1/40. However, larger networks are not more precise than smaller networks, and the performance of the large (1 801 and 15 601 parameters) networks with 126 test functions is even worse than the performance of the same network architectures trained with 70 test functions, especially in the primal formulation. In both cases, the training loss of the larger networks was orders of magnitude lower than the training loss of the smaller networks, see Figure 10 and Figure 11. This, however, does not translate into better estimates.

Training loss of the primal and dual solutions, and the gap between the estimates

Figure 10. Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 70 spectral test functions (VSPINN).

Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 126 spectral test functions (VSPINN)

Figure 11. Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 126 spectral test functions (VSPINN).

Because the VPINNs use the same architecture as PINNs, the pointwise strong-form residual can be calculated for comparison, shown in Figure 12 and Figure 13. The results suggest that the larger networks are prone to overfitting to the inaccuracy in the integration of the scalar products with the test functions in (14) and (15), especially considering the distortion in the solution with 15 601 parameters. This problem is worse for the basis of 126 test functions, as it contains functions with higher frequencies, for which the same integration scheme yields progressively lower precision. Integration problems might be remedied with different integration techniques, such as Filon’s quadrature [5], although these are ultimately out of the scope of this paper.

Primal solutions and the residuals of select VSPINNs depending on the number of the spectral test functions $N_t$

Figure 12. Primal solutions and the residuals of select VSPINNs depending on the number of the spectral test functions NtN_t.

Dual solutions and residuals of select VSPINNs

Figure 13. Dual solutions and the residuals of select VSPINNs depending on the number of the spectral test functions NtN_t.

While the weak formulation improves the performance of the smaller networks, allowing them to handle the piece-wise constant material without artificial smoothing, VSPINNs cannot compete with the FEM benchmark. Classic PINNs provide better results for larger networks, even considering the material approximation error. In terms of computational efficiency, there is no clear difference between VPINNs and classic PINNs, as VPINNs eliminate the need to compute the second-order gradients, but at the same time, they require more memory to process the integration with the test functions.

VPINN with neural network test functions (VNPINN). Due to the limitations of spectral basis functions, we explore other options for periodic functions, specifically neural networks. For each considered architecture, several instances are initialised with random weights, see examples in Figure 14.

Figure 14. Example of 10 randomly initialized neural network test functions based on the PINN architecture.

The benefits of this approach are twofold: (i) the functions are naturally from the same function space as the solution; (ii) more test functions can be added without worsening the precision of integration. The main downside is that the linear independence of these test functions is not guaranteed, mainly because of the weight-space symmetry of the neural networks [3]. The generally non-diagonal Gram matrix and its inverse in (22) have to be computed numerically, and we commonly observe that for the number of test functions approaching the number of neural network parameters, this matrix is no longer positive definite, typically with a few very small negative eigenvalues. Another option would be to construct an orthogonal basis by training the networks, but this is very time-consuming in practice. Instead, when the full Gram matrix was ill-conditioned, particularly for the network with 65 DoFs and Nt=100N_t = 100 or 200, we set all its non-diagonal values to zero.

Four differently sized PINN architectures using Nt={50,100,200}N_t = \{50, 100, 200\} were trained. The networks with 1 801 and 15 601 parameters were additionally trained with 400 test functions. The primal and dual estimates, as well as the guaranteed bounds, are shown in Figure 15.

Precision of the primal and dual estimates of the VNPINN depending on the number of network parameters and the number of neural network test functions

Figure 15. Precision of the primal and dual estimates of the VNPINN depending on the number of network parameters and the number of neural network test functions.

Here, we observe that both primal and dual solutions improve with the increasing number of parameters and test functions. Increasing the number of test functions for the smaller networks (65 and 391 parameters) has diminishing returns, as these networks perform similarly with 100 and 200 test functions. On the other hand, the precision of larger networks does not improve if the number of test functions does not increase accordingly. For example, the VNPINNs with 15 601 parameters trained with 50 test functions yield worse effective conductivity estimates and have higher strong-form residuals than the VNPINNs with 65 parameters trained with the same number of test functions. Since the achieved training loss of the large network is generally lower, the distortions in the solutions in Figure 18 and Figure 19 for the solution with 15 601 parameters and Nt=50N_t = 50 suggest the overfitting to the test functions. While this negative effect is less pronounced when the number of test functions is large (i.e., Nt∈200,400N_t \in200, 400), the comparison between the training loss and the primal-dual gap in Figure 16 and Figure 17 shows that while the convergence in training loss has not been reached in the allocated 40 000 epochs, the primal-dual gap has reached convergence around 30 000-th epoch, which, again, suggests the overfitting to the test functions.

Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 200 neural network test functions

Figure 16. Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 200 neural network test functions (VNPINN).

Training loss of the primal and dual solutions and the gap between the estimates

Figure 17. Training loss of the primal and dual solutions, and the gap between the estimates, VPINN with 200 neural network test functions (VNPINN).

Primal solutions and residuals of select VNPINNs

Figure 18. Primal solutions (top) and the residuals (bottom) of select VNPINNs depending on the number of the neural network test functions NtN_t.

Dual solutions (top) and residuals (bottom) of select VNPINNs depending on the number of neural network test functions $N_t$

Figure 19. Dual solutions (top) and the residuals (bottom) of select VNPINNs depending on the number of the neural network test functions NtN_t.

From the presented results, we conclude that the large networks (1 801 and 15 601 parameters) gain around 1% precision of the primal estimate for each doubling of the test basis, and up to 4% in the dual estimate. The point of diminishing returns is reached around 400 test functions, where the only improvement is in the primal solution by the largest network. This is explained by the overall slowdown in training for such a large number of test functions (Figure 17), compared to the smaller test basis (Figure 16).

Multiple Inclusions

The performance of strong-form PINNs in the primal formulation was thoroughly tested by Jiang et al. (2023) on multi-inclusion composites with varying volume fraction, where it was shown to achieve the same precision in the homogenized parameters as FEM. However, the aforementioned study did not consider the error introduced by the material smoothing. Here, we assume a composite with four randomly distributed equi-sized inclusions in a 2π×2π2\pi\times2\pi periodic cell, where γmat=0.1\gamma_{\mathrm{mat}} = 0.1 and γinc=1\gamma_{\mathrm{inc}} = 1 are thermal conductivities of the matrix and the inclusions, respectively, see Figure 20. For the PINN solution, we assume the smooth approximation of the conductivity function as

Smooth approximation of the material with multiple inclusions

Figure 20. Smooth approximation (a) As,ε(x)A_{\mathrm{s},\varepsilon}(\boldsymbol{x}) of the material with multiple inclusions (b)

As,ε(x)=(0.55−0.45∏i=14tanh⁡[1ε(∥x−xic∥22−r2)])I,(28)A_{\mathrm{s},\varepsilon}(\boldsymbol{x}) = \left(0.55 - 0.45 \prod_{i=1}^{4} \tanh\left[\frac{1}{\varepsilon}\left(\lVert\boldsymbol{x} - \boldsymbol{x}_{i}^{c}\rVert_{2}^{2} - r^{2}\right)\right]\right)I, \tag*{(28)}

where xic\boldsymbol{x}_{i}^{c} are the centers of the inclusions, rr is the radius of the inclusions, and ε>0\varepsilon> 0 controls the sharpness of the transition. Since the previous example with a single inclusion showed that there is an optimal ε\varepsilon that minimizes the material approximation error while still allowing the PINN to converge to a high-quality solution, we only test the PINNs on one such ε\varepsilon, which we found to be roughly equal to 0.06 of the inclusion radius 0.2π0.2\pi, or 0.012π0.012\pi.

As there is no known analytical solution for this problem, we compare the neural network solutions only to the FEM benchmark computed on a 200×200200 \times200 regular mesh, which provides the upper and lower bounds within 0.35%0.35\% of each other, see Figure 21. We then test two larger configurations of the neural networks (Table 2) in the primal and dual formulation as PINNs with the smooth material approximations and VNPINNs with 400 test functions. The collocation points are selected on the 200×200200 \times200 regular grid, which is then used for trapezoidal integration. All networks were trained for 100 000 epochs using the Adam optimizer and the initial learning rate of 10−510^{-5}.

Benchmark primal and dual solutions

Figure 21. Benchmark primal and dual solutions for ξ=[1,0]T\xi= [1,0]^{\mathrm{T}}, obtained with linear triangular finite elements, using discretization into 200×200200 \times200 unique DoFs.

Nº neurons, periodic layerNº neurons, per residual layerNº residual layersNº parameters
5050515 601
7070635 211

Table 2. Configurations of the considered neural networks for both primal versus dual and strong versus weak formulations, for the multiple inclusions problem.

The estimates and bounds of the homogenized parameter A1,1∗A_{1,1}^{*} from the primal and dual solutions for PINNs and VPINNs are shown in Figure 22, normalized by the dual FEM bound.

Precision of the primal and dual estimates of the PINN and VNPINN depending on the number of network parameters

Figure 22. Precision of the primal and dual estimates of the PINN and VNPINN depending on the number of network parameters.

Both 15 601 and 35 211-parameter PINN converged to virtually the same parameter estimate in the primal and dual formulation. However, when the material approximation error is considered, PINNs provide an estimate around 2% higher than the FEM benchmark, while the calculated bounds on the exact material distribution are up to 1.2% away from the FEM benchmark. Even with the material approximation error, PINNs have achieved a closer result to the benchmark than the VNPINNs that have been trained on the exact material distribution, whose primal bounds and estimates come to around 2.5% higher than the benchmark, and dual bounds and estimates are 0.5% lower. The PINN and VNPINN solutions and their residuals can be compared in Figure 23 and Figure 24. Interestingly, unlike in the previous example, the dual formulation provided a more accurate estimate. One possible explanation could lie in the opposite contrast ratios of selected conductivities in the matrix and inclusion in the two examples. The dependence on the relative conductivity might be corroborated by the observation that the strong-form residuals of the dual VPINNs are significantly higher for the less conductive (i.e., more resistant) square inclusion in comparison to the matrix in Figure 19, while more conductive circular inclusions have higher residuals in the primal formulation in Figure 24. Strong-form PINNs typically only have significantly higher residuals at the interface of the materials. The behavior of VPINNs for different material contrasts should be explored in a future study.

Primal solutions (top) and the residuals (bottom) of PINNs and VPINNs for the multiple inclusion problem

Figure 23. Primal solutions (top) and the residuals (bottom) of PINNs and VPINNs for the multiple inclusion problem.

Dual solutions and residuals of PINNs and VPINNs for the multiple inclusion problem

Figure 24. Dual solutions (top) and the residuals (bottom) of PINNs and VPINNs for the multiple inclusion problem.

The training of PINNs and VPINNs is shown in Figure 25 and Figure 26. The VPINN estimates stopped improving after about 20 000 epochs, which suggests the overfitting to the test functions, while PINN estimates achieved a similar precision slightly slower (around 30 000 epochs) but continued to improve significantly. However, considering that material smoothing puts a hard limit on the guaranteed bounds, the utility of such a prolonged training of PINNs should be assessed.

Training loss of the primal and dual solutions, and the gap between the estimates, PINN, multiple inclusion problem

Figure 25. Training loss of the primal and dual solutions, and the gap between the estimates, PINN, multiple inclusion problem.

Training loss of the primal and dual solutions, and the gap between the estimates, VNPINN, multiple inclusion problem

Figure 26. Training loss of the primal and dual solutions, and the gap between the estimates, VNPINN, multiple inclusion problem.

Discussion

The relative error of the final primal and dual estimates for all networks of the single inclusion experiment (§ 4.1) is summarised in Table 3. Table 4 then lists the computed guaranteed error bounds from the piecewise linear approximations of the solutions.

FormulationPrimalDual
N. of parameters653911 80115 601653911 80115 601
PINN, ε=1/10\varepsilon= 1/101.9441.4421.4411.4410.1671.4241.4401.441
PINN, ε=1/20\varepsilon= 1/2010.7720.6950.6940.694-12.3120.5440.6710.693
PINN, ε=1/30\varepsilon= 1/306.4590.4610.4560.457-26.174-7.417-0.1840.448
PINN, ε=1/40\varepsilon= 1/4015.51820.10319.68611.507-24.564-16.308-44.057-32.990
VSPINN, Nt=70N_t = 702.1362.7132.3333.054-3.819-7.229-5.820-6.388
VSPINN, Nt=126N_t = 1261.9451.7794.7347.054-5.023-6.010-7.937-6.091
VNPINN, Nt=50N_t = 502.2431.7122.1022.247-8.288-4.722-5.786-4.527
VNPINN, Nt=100N_t = 1001.9751.5831.3451.543-4.889-3.021-1.889-1.902
VNPINN, Nt=200N_t = 2001.9221.5770.5910.677-4.889-2.817-0.509-0.478
VNPINN, Nt=400N_t = 400——0.7390.460——-1.035-2.059

Table 3. Relative error (%) of the primal and dual estimates of the homogenized parameter for differently trained networks, best result in each group is in bold.

FormulationPrimalDual
N. of parameters653911 80115 601653911 80115 601
PINN, ε=1/10\varepsilon= 1/101.1900.7170.7180.718-4.922-4.465-4.490-4.493
PINN, ε=1/20\varepsilon= 1/2010.4600.3590.3590.359-12.883-2.068-2.065-2.064
PINN, ε=1/30\varepsilon= 1/306.2040.2460.2400.244-26.218-7.990-1.608-1.168
PINN, ε=1/40\varepsilon= 1/4014.72918.72719.50111.195-24.578-18.093-45.371-33.549
VSPINN, Nt=70N_t = 702.0992.6922.2983.033-5.013-8.569-7.003-7.734
VSPINN, Nt=126N_t = 1261.9121.7494.7077.035-6.288-7.183-8.883-7.551
VNPINN, Nt=50N_t = 502.2111.6632.0802.226-9.151-5.944-7.319-6.272
VNPINN, Nt=100N_t = 1001.9591.5541.3121.514-6.069-4.089-3.210-3.282
VNPINN, Nt=200N_t = 2001.9051.5500.5370.605-6.069-4.388-2.053-1.542
VNPINN, Nt=400N_t = 400——0.6970.406——-2.542-3.317

Table 4. Relative error (%) of guaranteed primal and dual bounds of the homogenized parameter for differently trained networks, best result in each group is in bold.

As shown in Table 4, the strong-form PINNs demonstrate superior convergence for smoothly approximated materials with ε∈{1/10,1/20,1/30}\varepsilon\in\{1/10, 1/20, 1/30\}, achieving relative errors below 0.5% for the largest networks in the primal formulation. However, this performance degrades catastrophically when ε=1/40\varepsilon= 1/40, with relative errors exceeding 10% even for the 15 601-parameter network, indicating a critical failure mode when the material transition becomes too sharp.

Among the variational approaches, VNPINNs with neural network test functions outperform VSPINNs for most configurations examined. The VSPINNs exhibit non-monotonic behavior with respect to network size, as evidenced by the degraded performance of the 15,601-parameter network with 126 test functions (7.054% error) compared to smaller networks. This suggests overfitting to integration errors when high-frequency spectral basis functions are employed with insufficient quadrature precision. In contrast, VNPINNs show more consistent improvement with increasing the number of test functions, achieving their best performance with 400 test functions for the largest networks, reaching 0.460% relative error in the primal formulation.

The guaranteed bounds presented in Table 4 are close to the estimates, with the projection onto piecewise linear finite elements introducing minimal additional error (typically less than 0.5%). With precise integration, the bounds calculated from the projected solution maintain the theoretical ordering property of Eq. (18) for all solutions, providing a reliable indicator of solution quality. Unfortunately, this requires a careful meshing of the geometry, which is not always feasible and diminishes one of the primary benefits of PINNs. However, in practice, a reasonably precise numerical integration typically produces reliable results as well, as demonstrated by the primary estimates that can be computed fast during the training of the network.

For the multiple inclusions example, we similarly conclude that strong-form PINNs can perform extremely well for smooth geometries, but cannot match even simple piece-wise linear FEM for discontinuous materials. The quality of the test basis remains the main limiting factor for VPINNs, and the number of required test functions depends on the complexity of the geometry as well.

Conclusion

This work demonstrates the integration of dual formulation principles from classical homogenization theory into the PINN framework for computing effective properties of thermo-conductive composites. The dual approach provides not only theoretically guaranteed bounds on homogenized parameters but also serves as a practical diagnostic tool for identifying convergence failures that might otherwise go undetected in standard PINN implementations.

We examined the effectiveness of dual formulations within two primary strategies for addressing PINN limitations with discontinuous materials: (i) smooth approximation of material coefficients in strong-form PINNs, and (ii) variational formulations with both spectral and neural network test function bases. Our findings reveal fundamental trade-offs between these approaches. Strong-form PINNs achieve superior accuracy for sufficiently smooth material approximations but require careful calibration of the smoothing parameter to avoid catastrophic(and unidentifiable without the duality gap calculation ) failure. VPINNs, though computationally more demanding and sensitive to test function selection, offer greater robustness by directly accommodating discontinuous material properties.

The primal-dual framework enhances the reliability of PINN-based homogenization by providing complementary solutions that bound the true effective properties. The gap between primal and dual estimates serves as an intrinsic quality metric, enabling practitioners to assess solution reliability without reference solutions.

Looking forward, extending this dual formulation approach to three-dimensional problems and linear elasticity presents both challenges and opportunities. The divergence-free constraint in 3D cannot be represented through a scalar stream function as in 2D, necessitating modified network architectures and additional training loss terms. However, classical numerical methods face similar increases in complexity in their dual formulations, requiring significantly more degrees of freedom than their primal counterparts. This parallel suggests that PINN-based approaches may retain competitive advantages even as problem complexity increases, particularly given their mesh-free nature and potential for rapid evaluation once trained.

Implementation and trained neural networks are available on GitHub.

CRediT Author Statement

LG: Conceptualization, Methodology, Software, Investigation, Writing – Original Draft, Visualization, Funding Acquisition; OR: Methodology, Writing – Review & Editing, Supervision; MD: Methodology, Writing – Review & Editing, Supervision; IP: Methodology, Writing – Review & Editing, Supervision, Funding Acquisition.

Acknowledgments

LG and IP acknowledge funding by the European Union under the project ROBOPROX (reg. no. CZ.02.01.01/00/22_008/0004590). MD’s work was supported by the Czech Science Foundation, project No. 24-11845S. LG also acknowledges the Student Grant Competition of the Czech Technical University in Prague (project no. SGS25/002/OHK1/1T/11).

References

References

  1. [1]Belytschko, T., Gracie, R., Ventura, G., 2009. A review of extended/generalized finite element methods for material modeling. Modelling and Simulation in Materials Science and Engineering 17, 043001. doi:10.1088/0965-0393/17/4/043001.DOI
  2. [2]Berrone, S., Canuto, C., Pintore, M., 2022. Variational physics informed neural networks: the role of quadratures and test functions. Journal of Scientific Computing 92. doi:10.1007/s10915-022-01950-4.DOI
  3. [3]Brea, J., Simsek, B., Illing, B., Gerstner, W., 2019. Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape. doi:10.48550/arXiv.1907.02911, arXiv:1907.02911.DOI
  4. [4]Briane, M., Manceau, D., 2008. Duality results in the homogenization of two-dimensional high-contrast conductivities. Networks and Heterogeneous Media 3, 509–522.DOI
  5. [5]Chase, S.M., Fosdick, L.D., 1969. An algorithm for Filon quadrature. Commun. ACM 12, 453–457. doi:10.1145/363196.363209.DOI
  6. [6]Davoli, E., Kru\uží̄k, M., Pagliari, V., 2024. Homogenization of high-contrast composites under differential constraints. Advances in Calculus of Variations 17, 277–318. doi:doi:10.1515/acv-2022-0009.
  7. [7]E, W., Yu, B., 2018. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics 6, 1–12. doi:10.1007/s40304-018-0127-z.DOI
  8. [8]Engquist, B., Souganidis, P.E., 2008. Asymptotic and numerical homogenization. Acta Numerica 17, 147–190. doi:10.1017/S0962492906360011.DOI
  9. [9]Farkane, A., Ghogho, M., Oudani, M., Boutayeb, M., 2023. EPINN-NSE: Enhanced physics-informed neural networks for solving Navier-Stokes equations. doi:10.48550/arXiv.2304.03689, arXiv:2304.03689.DOI
  10. [10]Gaynudinova, L., Ladecký, M., Nekvinda, A., Pultarová, I., Zeman, J., 2022. Efficient numerical method for reliable upper and lower bounds on homogenized parameters. doi:10.48550/arXiv.2208.09940.
  11. [12]Geers, M., Kouznetsova, V., Brekelmans, W., 2010. Multi-scale computational homogenization: Trends and challenges. Journal of Computational and Applied Mathematics 234, 2175–2182. doi:10.1016/j.cam.2009.08.077.DOI
  12. [13]Girault, V., Raviart, P.A., 1979. Finite Element Approximation of the Navier-Stokes Equations. Springer Berlin Heidelberg. doi:10.1007/bfb0063447.DOI
  13. [14]Grossmann, T.G., Komorowska, U.J., Latz, J., Schönlieb, C.B., 2024. Can physics-informed neural networks beat the finite element method? IMA Journal of Applied Mathematics 89, 143–174. doi:10.1093/imamat/hxae011.DOI
  14. [15]Haslinger, J., Dvořák, J., 1995. Optimum composite material design. ESAIM: Mathematical Modelling and Numerical Analysis 29, 657–686.DOI
  15. [16]Henkes, A., Wessels, H., Mahnken, R., 2022. Physics-informed neural networks for continuum micromechanics. Computer Methods in Applied Mechanics and Engineering 393, 114790. doi:10.1016/j.cma.2022.114790.DOI
  16. [17]Jagtap, A.D., Karniadakis, G.E., 2020. Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics 28, 2002–2041. doi:10.4208/cicp.oa-2020-0164.DOI
  17. [18]Jiang, J., Wu, J., Chen, Q., Chatzigeorgiou, G., Meraghni, F., 2023. Physically informed deep homogenization neural network for unidirectional multiphase/multi-inclusion thermoconductive composites. Computer Methods in Applied Mechanics and Engineering 409, 115972. doi:10.1016/j.cma.2023.115972.DOI
  18. [19]Jikov, V.V., Kozlov, S.M., Oleinik, O.A., 2012. Homogenization of differential operators and integral functionals. Springer Science & Business Media.
  19. [20]Kharazmi, E., Zhang, Z., Karniadakis, G.E., 2019. Variational physics-informed neural networks for solving partial differential equations. doi:10.48550/arXiv.1912.00873, arXiv:1912.00873.DOI
  20. [21]Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Mahoney, M.W., 2021. Characterizing possible failure modes in physics-informed neural networks. Advances in neural information processing systems 34, 26548–26560. doi:10.5555/3540261.3542294.arxiv.org/abs/2109.01050
  21. [22]Legrain, G., Cartraud, P., Perreard, I., Moës, N., 2011. An X-FEM and level set computational approach for image-based modelling: Application to homogenization. International Journal for Numerical Methods in Engineering 86, 915–934. doi:10.1002/nme.3085.DOI
  22. [23]Lei, G., Exposito, D., Mao, X., 2025. Discontinuity-aware KAN-based physics-informed neural networks. doi:10.48550/arXiv.2507.08338, arXiv:2507.08338.DOI
  23. [24]Leung, W.T., Lin, G., Zhang, Z., 2022. NH-PINN: Neural homogenization-based physics-informed neural network for multiscale problems. Journal of Computational Physics 470, 111539. doi:10.1016/j.jcp.2022.111539.DOI
  24. [25]LeVeque, R.J., Li, Z., 1994. The immersed interface method for elliptic equations with discontinuous coefficients and singular sources. SIAM Journal on Numerical Analysis 31, 1019–1044. doi:10.1137/0731054.DOI
  25. [26]Liu, L., Liu, S., Xie, H., Xiong, F., Yu, T., Xiao, M., Liu, L., Yong, H., 2023. Discontinuity computing using physics-informed neural networks. Journal of Scientific Computing 98. doi:10.1007/s10915-023-02412-1.DOI
  26. [27]Michel, J., Moulinec, H., Suquet, P., 1999. Effective properties of composite materials with periodic microstructure: a computational approach. Computer Methods in Applied Mechanics and Engineering 172, 109–143. doi:10.1016/S0045-7825(98)00227-8.DOI
  27. [28]Milton, G., 2022. The Theory of Composites. Classics in Applied Mathematics, Society for Industrial and Applied Mathematics, SIAM.
  28. [29]Mishra, S., Molinaro, R., 2023. Estimates on the generalization error of physics informed neural networks (PINNs) for approximating PDEs. doi:10.48550/arXiv.2006.16144, arXiv:2006.16144.DOI
  29. [30]Moulinec, H., Suquet, P., 1994. A fast numerical method for computing the linear and non-linear mechanical properties of composites. Comptes Rendus de l’Académie des sciences. Série II. Mécanique, physique, chimie, astronomie 318, 1417–1423.
  30. [31]Obnosov, Y.V., 1999. Periodic heterogeneous structures: New explicit solutions and effective characteristics of refraction of an imposed field. SIAM Journal on Applied Mathematics 59, 1267–1287. doi:10.1137/S0036139997314770.DOI
  31. [32]Park, J.S.R., Zhu, X., 2022. Physics-informed neural networks for learning the homogenized coefficients of multiscale elliptic equations. Journal of Computational Physics 467, 111420. doi:10.1016/j.jcp.2022.111420.DOI
  32. [33]Raissi, M., Perdikaris, P., Karniadakis, G., 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, 686–707. doi:10.1016/j.jcp.2018.10.045.DOI
  33. [34]Richter-Powell, J., Lipman, Y., Chen, R.T.Q., 2022. Neural conservation laws: A divergence-free perspective. doi:10.48550/arXiv.2210.01741, arXiv:2210.01741.
  34. [35]Rodrigues, D., Belinha, J., Pires, F., Dinis, L., Jorge, R.N., 2018. Homogenization technique for heterogeneous composite materials using meshless methods. Engineering Analysis with Boundary Elements 92, 73–89. doi:10.1016/j.enganabound.2017.12.012. improved Localized and Hybrid Meshless Methods - Part 1.DOI
  35. [36]Rojas, S., Maczuga, P., Muñoz-Matute, J., Pardo, D., Paszyński, M., 2024. Robust variational physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 425, 116904. doi:10.1016/j.cma.2024.116904.DOI
  36. [37]Ryck, T.D., Jagtap, A.D., Mishra, S., 2023. Error estimates for physics informed neural networks approximating the Navier-Stokes equations. doi:10.48550/arXiv.2203.09346, arXiv:2203.09346.arxiv.org/abs/2203.09346
  37. [38]Schneider, M., 2021. A review of nonlinear FFT-based computational homogenization methods. Acta Mechanica 232, 2051–2100.DOI
  38. [39]Soyarslan, C., Pradas, M., 2024. Physics-informed machine learning in asymptotic homogenization of elliptic equations. Computer Methods in Applied Mechanics and Engineering 427, 117043. doi:10.1016/j.cma.2024.117043.DOI
  39. [40]Vondřejc, J., Zeman, J., Marek, I., 2015. Guaranteed upper–lower bounds on homogenized properties by FFT-based Galerkin method. Computer Methods in Applied Mechanics and Engineering 297, 258–291. doi:10.1016/j.cma.2015.09.003.DOI
  40. [42]Yvonnet, J., 2019. Computational homogenization of heterogeneous materials with finite elements. volume 258. Springer.DOI
  41. [43]Zhao, L., Shao, Q., 2025. DENNs: Discontinuity-embedded neural networks for fracture mechanics. Computer Methods in Applied Mechanics and Engineering 446, 118184. doi:10.1016/j.cma.2025.118184.DOI

Paper details

vlnc:W_zx2ksw2z
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents