betaFeedback

WorksPhysics and Astronomy

A variational neural network approach for glacier modelling with nonlinear rheology

Abstract

In this paper, we propose a mesh-free method to solve full stokes equation which models the glacier movement with nonlinear rheology. Our approach is inspired by the Deep-Ritz method proposed in [12]. We first formulate the solution of non-Newtonian ice flow model into the minimizer of a variational integral with boundary constraints. The solution is then approximated by a deep neural network whose loss function is the variational integral plus soft constraint from the mixed boundary conditions. Instead of introducing mesh grids or basis functions to evaluate the loss function, our method only requires uniform samplers of the domain and boundaries. To address instability in real-world scaling, we re-normalize the input of the network at the first layer and balance the regularizing factors for each individual boundary. Finally, we illustrate the performance of our method by several numerical experiments, including a 2D model with analytical solution, Arolla glacier model with real scaling and a 3D model with periodic boundary conditions. Numerical results show that our proposed method is efficient in solving the non-Newtonian mechanics arising from glacier modeling with nonlinear rheology.

AMS subject classification: 35A15, 65J15, 68T99, 70K25, 76A05.

Introduction

In recent years, deep neural networks (DNNs) have achieved unprecedented levels of success in a broad range of areas such as computer vision, speech recognition, natural language processing, and health sciences, producing results comparable or superior to human experts [29, 16]. The impacts have reached physical sciences where traditional first-principle based modeling and computational methodologies have been the norm. Thanks in part to the user-friendly open-source computing platforms from industry (e.g. TensorFlow and PyTorch), there have been vibrant activities in applying deep learning tools for scientific computing, such as approximating multivariate functions, solving ordinary/partial differential equations (ODEs/PDEs) and inverse problems using DNNs; see, e.g. [49, 12, 18, 26, 2, 62, 57] and references therein. There are many classical works on the approximation power of neural networks; see e.g. [10, 21, 13, 45]. For recent works on the expressive (approximation) power of DNNs; see, e.g. [9, 52, 60, 39, 34, 53]. In [18], the authors showed that DNNs with rectified linear unit (ReLU) activation function and enough width/depth contain the continuous piece-wise linear finite element space. Thus, one can represent a solution of PDE using the ReLU-DNN.

Solving ODEs or PDEs by a neural network (NN) approximation is known in the literature dating back at least to the 1990’s; see e.g. [30, 38, 28]. The main idea in these works is to train NNs to approximate the solution by minimizing the residual of the ODEs or PDEs, along with the associated initial and boundary conditions. These early works estimate neural network solutions on a fixed mesh. Recently DNN methods are developed for Poisson and eigenvalue problems with a variational principle characterization (deep Ritz, [12]), for a class of high-dimensional parabolic PDEs with stochastic representations [17], for advancing finite element methods [19, 8, 7], for nonconvex energy minimization in simulating martensitic phase transitions [? ], and for learning and generating invariant measures of stochastic dynamical systems with parameters [? ]. The physics-informed neural network (PINN) method [47] and a deep Galerkin method (DGM) [54] compute PDE solutions based on their physical properties. For parametric PDEs, a deep operator network (DeepONet) learns operators accurately and efficiently from a relatively small dataset based on the universal approximation theorem of operators [35]; a Fourier neural operator method [31] directly learns the mapping from functional parametric dependence to the solutions of a family of PDEs. In [61, 1], weak adversarial network methods are studied for weak solutions and inverse problems, see also related studies on PDE recovery from data via DNN [33, 32, 46, 58] among others. In the context of surrogate modeling and uncertainty quantification (UQ), DNN methods include Bayesian deep convolutional encoder-decoder networks [62], deep multi-scale model learning [56], physics-constrained deep learning method [63], see also [26, 52, 25, 59] and references therein.

In this work, we present a deep learning method for solving problems in non-Newtonian mechanics that obey certain variational principles. In particular, we focus on nonlinear Stokes problems in which the viscosity nonlinearly depends on the strain rate. This type of problems plays a fundamental role in modelling geodynamic processes, for instance, the dynamics of glaciers [22, 44] and mantle convection [50, 37]. The solutions of these problems typically face a combination of challenges, such as the presence of local features emerging from the nonlinear rheology, saddle point problems due to the incompressibility condition, complex problem domain with high aspect ratios, and high contrast boundary conditions. There has been a lot of effort in developing efficient numerical methods to address these challenges. For example, the work of [5, 14, 20, 23, 51, 55] designed and analyzed efficient and accurate high-order finite element discretization schemes. Dedicated adaptation strategies, see [23, 48] and references therein, are developed to ensure that these high-order methods can successfully resolve local features of the flow field and a wide range of length scales. The resulting discretized nonlinear systems have to be solved using either Picard fixed-point iterations or Newton’s method. Most of the recent high-performance solvers adopt modified Newton’s iterations together with high-performance iterative linear solvers, see, e.g., [6, 15, 23, 36], to obtain a superlinear convergence rate.

Our work is inspired by the deep Ritz method proposed in [12] and our recent progress in developing deep learning method to solve interface problem [57]. We first formulate the PDEs into a variational problem, which leads to the objective function used by the neural network training. The second part consists of penalty terms arising from the boundary constraint of the governing PDE. In real world models, e.g. glacier sliding, the scale of the problem domain and the scale of physical parameters vary significantly. To address these challenges, we introduce a normalizing layer following the input of neural networks and design a strategy to balance penalty factors due to different boundary conditions. Using this combination of strategies, our proposed deep learning method is capable to solve nonlinear Stokes problems with different geometric scales, different parameter scales and boundary conditions using a universal configuration of network parameters.

The rest of the paper is organized as follows. In § 2, we introduce the background of the Non-Newtonian ice flow model. In § 3, we review the basic idea of deep neural network and formulate the variational neural network approach for solving the Non-Newtonian ice flow model. In addition, we also discuss some details of the implementation of our method. In § 4, we present numerical results to demonstrate the accuracy of our method. § 5 offers some concluding remarks.

Background

Since glaciers form one of the natural low-pass filters of atmospheric variability, modelling the mechanics of glaciers is instrumental in revealing slow changes in the climate system that might otherwise be obscured by short-term noise. We consider a canonical glacier model that treats the flow of ice as non-Newtonian, viscous, incompressible, and isothermal fluid in the steady-state [11]. In this section, we first present the strong form of the ice flow model, and then discuss its variational formulation that naturally yields the weak form of the ice flow model.

Non-Newtonian ice flow model

For an open, bounded domain Ω⊂Rd\Omega\subset\mathbb{R}^{d}, we denote the velocity field (measured in meters per calendar year, i.e., m a−1^{-1}) and the stress tensor (measured in Pa) of ice flow by u=(u1,…,ud)T:Ω↦Rd\mathbf{u} = (u_{1}, \ldots, u_{d})^{\mathsf{T}} : \Omega\mapsto\mathbb{R}^{d} and σu:Ω↦Rd×d\boldsymbol{\sigma}_{\mathbf{u}} : \Omega\mapsto\mathbb{R}^{d \times d}, respectively. The conservation laws of momentum and mass state that

−∇⋅σu=ρg,(1)-\nabla\cdot\boldsymbol{\sigma}_{\mathbf{u}} = \rho\mathbf{g}, \tag*{(1)}
∇⋅u=0,(2)\nabla\cdot\mathbf{u} = 0, \tag*{(2)}

where ρ\rho is the ice density (910 kg m−3^{-3}) and g\mathbf{g} is gravitational acceleration (9.81 m s−2^{-2}, where s represents a second). The stress tensor is split into a deviatoric part and an isotropic pressure pp, i.e., σu=τu−pI\boldsymbol{\sigma}_{\mathbf{u}} = \boldsymbol{\tau}_{\mathbf{u}} - p\mathbf{I}. Denoting the strain rate tensor of the velocity field by

ε˙u=12(∇u+∇uT),(3)\dot{\boldsymbol{\varepsilon}}_{\mathbf{u}} = \frac{1}{2}\left(\nabla\mathbf{u} + \nabla\mathbf{u}^{\mathsf{T}}\right), \tag*{(3)}

the Glen’s law of rheology [40] links the deviatoric stress to the strain rate via the constitutive equation

τu=2η(u)ε˙uwithη(u)=12A−1n(12ε˙u:ε˙u)1−n2n,(4)\boldsymbol{\tau}_{\mathbf{u}} = 2\eta(\mathbf{u})\dot{\boldsymbol{\varepsilon}}_{\mathbf{u}} \quad\text{with} \quad\eta(\mathbf{u}) = \frac{1}{2}A^{-\frac{1}{n}}\left(\frac{1}{2}\dot{\boldsymbol{\varepsilon}}_{\mathbf{u}} : \dot{\boldsymbol{\varepsilon}}_{\mathbf{u}}\right)^{\frac{1-n}{2n}}, \tag*{(4)}

where η(u)\eta(\mathbf{u}) is the effective viscosity nonlinearly depending on the strain rate and the double-dot product is the Frobenius inner product defined as

σ1:σ2=trace⁡(σ1Tσ2)=∑i,j(σ1)ij(σ2)ij,\boldsymbol{\sigma}_{1} : \boldsymbol{\sigma}_{2} = \operatorname{trace}\left(\boldsymbol{\sigma}_{1}^{\mathsf{T}}\boldsymbol{\sigma}_{2}\right) = \sum_{i,j}(\boldsymbol{\sigma}_{1})_{ij}(\boldsymbol{\sigma}_{2})_{ij},

for two second order tensors σ1\boldsymbol{\sigma}_{1} and σ2\boldsymbol{\sigma}_{2}. Here A=10−16(Pa−na−1)A = 10^{-16}(\mathrm{Pa}^{-n}\mathrm{a}^{-1}) is the flow parameter and we often set n=3n = 3. Using the nonlinear constitutive equation (4), the conservation of momentum (1) becomes

−∇⋅τu+∇p=ρg.(5)-\nabla\cdot\boldsymbol{\tau}_{\mathbf{u}} + \nabla p = \rho\mathbf{g}. \tag*{(5)}

Without loss of generality, we assume the domain Ω\Omega is bounded by two disjoint surfaces, Γb\Gamma_{b} the bottom boundary and Γt=∂Ω\Γb\Gamma_{t} = \partial\Omega\backslash\Gamma_{b} the top boundary. Let n\mathbf{n} denote the unit outward normal vector at any point on the boundary ∂Ω\partial\Omega. On the top boundary we impose the traction-free boundary condition

τun−p0n=0onΓt,(6)\boldsymbol{\tau}_{\mathbf{u}}\mathbf{n} - p_{0}\mathbf{n} = 0 \quad\text{on} \quad\Gamma_{t}, \tag*{(6)}

where p0p_{0} is the atmospheric pressure. Since the atmospheric pressure is often assumed to be negligible [42], so that we adopt p0=0p_{0} = 0 here. On the bottom boundary we impose a no-penetration condition along the outward normal direction, i.e.,

u⋅n=0onΓb,(7)\mathbf{u} \cdot\mathbf{n} = 0 \quad\text{on} \quad\Gamma_{b}, \tag*{(7)}

and a sliding boundary condition along the tangential direction. Given the map to the tangential direction T=I−n⊗n\mathbf{T}=\mathbf{I}-\mathbf{n}\otimes\mathbf{n}, the sliding boundary condition is given as

T(τun−pn)+βTu=0on Γb,\mathbf{T}(\boldsymbol{\tau}_{\mathbf{u}}\mathbf{n}-p\mathbf{n})+\beta\mathbf{T}\mathbf{u}=0 \quad\text{on }\Gamma_b,

for some basal drag coefficient β>0\beta>0. Note that the basal drag coefficient β\beta is a function of location in general. Since Tn=0\mathbf{T}\mathbf{n}=0, the sliding boundary condition does not depend on the pressure, which can be reduced to

T(τun)+βTu=0on Γb.(8)\mathbf{T}(\boldsymbol{\tau}_{\mathbf{u}}\mathbf{n})+\beta\mathbf{T}\mathbf{u}=0 \quad\text{on }\Gamma_b. \tag*{(8)}

Along the tangential direction, there is a special case that β→∞\beta\to\infty, where the sliding boundary condition becomes a non-sliding Dirichlet boundary condition, i.e., Tu=0\mathbf{T}\mathbf{u}=0. In later numerical examples, our algorithm can be adapted to both cases.

To summarize, the ice flow with nonlinear rheology can be modelled by a system of equations

−∇⋅τu+∇p=fon Ω∇⋅u=0on Ωτun−p0n=0on Γtu⋅n=0on ΓbT(τun)+βTu=0on Γb,(9)\begin{aligned} -\nabla\cdot\boldsymbol{\tau}_{\mathbf{u}}+\nabla p&=\mathbf{f} &&\text{on }\Omega\\ \nabla\cdot\mathbf{u}&=0 &&\text{on }\Omega\\ \boldsymbol{\tau}_{\mathbf{u}}\mathbf{n}-p_0\mathbf{n}&=0 &&\text{on }\Gamma_t \\ \mathbf{u}\cdot\mathbf{n}&=0 &&\text{on }\Gamma_b \\ \mathbf{T}(\boldsymbol{\tau}_{\mathbf{u}}\mathbf{n})+\beta\mathbf{T}\mathbf{u}&=0 &&\text{on }\Gamma_b, \tag*{(9)} \end{aligned}

where β>0\beta>0, f=ρg\mathbf{f}=\rho\mathbf{g}, T=I−n⊗n\mathbf{T}=\mathbf{I}-\mathbf{n}\otimes\mathbf{n}, and τu\boldsymbol{\tau}_{\mathbf{u}} given in (4). With β→∞\beta\to\infty, the mixed boundary conditions in (12) and (13) simply become u=0\mathbf{u}=0 on Γb\Gamma_b.

Variational formulation

The momentum equation (9) and the divergence-free mass equation (10) have the variational form: finding a vector function u∈H1(Ω)\mathbf{u}\in H^1(\Omega) satisfying boundary conditions (11) and (12) and a scalar function p∈L2(Ω)p\in L^2(\Omega) satisfying p=p0p=p_0 on Γt\Gamma_t such that

∫Ω(−∇⋅τu+∇p−f)⋅v dV=0(10)\int_{\Omega}(-\nabla\cdot\boldsymbol{\tau}_{\mathbf{u}}+\nabla p-\mathbf{f})\cdot\mathbf{v}\,dV=0 \tag*{(10)}
−∫Ωq∇⋅u dV=0,(11)-\int_{\Omega}q\nabla\cdot\mathbf{u}\,dV=0, \tag*{(11)}

for all vector functions v∈H1(Ω)\mathbf{v}\in H^1(\Omega) satisfying v⋅n=0\mathbf{v}\cdot\mathbf{n}=0 on Γb\Gamma_b and τvn−p0n=0\boldsymbol{\tau}_{\mathbf{v}}\mathbf{n}-p_0\mathbf{n}=0 on Γt\Gamma_t and all scalar function q∈L2(Ω)q\in L^2(\Omega) satisfying q=p0q=p_0 on Γt\Gamma_t. Applying the divergence theorem for vectors and tensors, the momentum equation (14) can also be expressed as

∫Ω(τu:ε˙v−p∇⋅v−f⋅v) dV−∮∂Ω(τun−pn)⋅v dS=0.(12)\int_{\Omega}(\boldsymbol{\tau}_{\mathbf{u}}:\dot{\boldsymbol{\varepsilon}}_{\mathbf{v}}-p\nabla\cdot\mathbf{v}-\mathbf{f}\cdot\mathbf{v})\,dV-\oint_{\partial\Omega}(\boldsymbol{\tau}_{\mathbf{u}}\mathbf{n}-p\mathbf{n})\cdot\mathbf{v}\,dS=0. \tag*{(12)}

The solutions (u,p)(\mathbf{u},p) satisfying the above variational form can be expressed as the minimizer of some energy functional subject to the divergence-free constraint (11) and some boundary conditions (11)–(13). This provides a starting point for applying deep neural network approximation to model the glacier flow. We present the energy functional and the corresponding optimization problem for the sliding and the the non-sliding boundary conditions as follows.

Sliding bottom boundary. Subject to the traction free boundary condition (11) on the top boundary, the boundary integral at the top boundary is eliminated, i.e.,

∮Γt(τn−pn)⋅v dS=∮Γt(τn−p0n)⋅v dS=0.(13)\oint_{\Gamma_t}(\boldsymbol{\tau}\mathbf{n}-p\mathbf{n})\cdot\mathbf{v}\,dS=\oint_{\Gamma_t}(\boldsymbol{\tau}\mathbf{n}-p_0\mathbf{n})\cdot\mathbf{v}\,dS=0. \tag*{(13)}

where p0p_0 is the atmospheric pressure at the top boundary. At the bottom boundary, applying the condition v⋅n=0\mathbf{v}\cdot\mathbf{n}=0 on Γb\Gamma_b and the sliding boundary condition (13) we have

∮Γb(τn−pn)⋅v dS=∮Γb(τn)⋅v dS=∮Γb(T(τn))⋅(Tv) dS+∮Γb((I−T)(τn))⋅((I−T)v) dS=−∮Γbβ(Tu)⋅(Tv) dS.(14)\begin{aligned} \oint_{\Gamma_b}(\boldsymbol{\tau}\mathbf{n}-p\mathbf{n})\cdot\mathbf{v}\,dS &=\oint_{\Gamma_b}(\boldsymbol{\tau}\mathbf{n})\cdot\mathbf{v}\,dS\\ &=\oint_{\Gamma_b}\bigl(\mathbf{T}(\boldsymbol{\tau}\mathbf{n})\bigr)\cdot(\mathbf{T}\mathbf{v})\,dS+\oint_{\Gamma_b}\bigl((\mathbf{I}-\mathbf{T})(\boldsymbol{\tau}\mathbf{n})\bigr)\cdot\bigl((\mathbf{I}-\mathbf{T})\mathbf{v}\bigr)\,dS\\ &=-\oint_{\Gamma_b}\beta(\mathbf{T}\mathbf{u})\cdot(\mathbf{T}\mathbf{v})\,dS. \tag*{(14)} \end{aligned}

as range⁡(T)\operatorname{range}(\mathbf{T}) and range⁡(I−T)\operatorname{range}(\mathbf{I}-\mathbf{T}) are mutually orthogonal and (I−T)v=0(\mathbf{I}-\mathbf{T})\mathbf{v}=0. Therefore, the boundary integral in (12) can be simplified as

−∮∂Ω(τn−pn)⋅v dS=∮Γbβ(Tu)⋅(Tv) dS.-\oint_{\partial\Omega}(\boldsymbol{\tau}\mathbf{n}-p\mathbf{n})\cdot\mathbf{v}\,dS=\oint_{\Gamma_b}\beta(\mathbf{T}\mathbf{u})\cdot(\mathbf{T}\mathbf{v})\,dS.

As shown in [11], the solutions (u,p)(\mathbf{u},p) satisfying the variational form (12) can be considered as the minimizer of the the energy functional

Es(u)=∫Ω(2n1+nA−1n(12ϵ˙u:ϵ˙u)1+n2n−ρg⋅u) dV+12∫Γbβ(Tu)⋅(Tu) dS.(15)\mathcal{E}_s(\mathbf{u})=\int_{\Omega}\left(\frac{2n}{1+n}A^{-\frac{1}{n}}\left(\frac{1}{2}\dot{\boldsymbol{\epsilon}}_{\mathbf{u}}:\dot{\boldsymbol{\epsilon}}_{\mathbf{u}}\right)^{\frac{1+n}{2n}}-\rho\mathbf{g}\cdot\mathbf{u}\right)\,dV+\frac{1}{2}\int_{\Gamma_b}\beta(\mathbf{T}\mathbf{u})\cdot(\mathbf{T}\mathbf{u})\,dS. \tag*{(15)}

subject to the divergence-free condition (10). Formulating this constrained optimization problem using the method of Lagrange multiplier, we have the Lagrangian functional

Ls(u,p)=Es(u)−∫Ωp∇⋅u dV.(16)\mathcal{L}_s(\mathbf{u},p)=\mathcal{E}_s(\mathbf{u})-\int_{\Omega}p\nabla\cdot\mathbf{u}\,dV. \tag*{(16)}

in which the pressure function pp plays the role of the Lagrange multiplier. Since the directional derivative of f(u):=12ϵ˙u:ϵ˙uf(\mathbf{u}):=\frac{1}{2}\dot{\boldsymbol{\epsilon}}_{\mathbf{u}}:\dot{\boldsymbol{\epsilon}}_{\mathbf{u}} along a function v\mathbf{v} is f′(u)[v]=ϵ˙u:ϵ˙vf'(\mathbf{u})[\mathbf{v}]=\dot{\boldsymbol{\epsilon}}_{\mathbf{u}}:\dot{\boldsymbol{\epsilon}}_{\mathbf{v}}, the variation of L(u,p)\mathcal{L}(\mathbf{u},p) along functions (v,q)(\mathbf{v},q) takes the form

Ls′(u,p)[v,q]=∫Ω(τu:ϵ˙v−p∇⋅v−ρg⋅v) dV−∫Ωq∇⋅u dV+∮Γbβ(Tu)⋅(Tv) dS.\mathcal{L}'_s(\mathbf{u},p)[\mathbf{v},q]=\int_{\Omega}(\boldsymbol{\tau}_{\mathbf{u} }:\dot{\boldsymbol{\epsilon}}_{\mathbf{v}}-p\nabla\cdot\mathbf{v}-\rho\mathbf{g}\cdot\mathbf{v})\,dV-\int_{\Omega}q\nabla\cdot\mathbf{u}\,dV+\oint_{\Gamma_b}\beta(\mathbf{T}\mathbf{u})\cdot(\mathbf{T}\mathbf{v})\,dS.

The solution to the Lagrange multiplier, (u,p)(\mathbf{u},p) such that Ls′(u,p)[v,q]=0\mathcal{L}'_{\mathrm{s}}(\mathbf{u},p)[\mathbf{v},q]=0 for all (v,q)(\mathbf{v},q) is equivalent to the solution to the variational form defined in (10) and (11). Thus, the velocity field u\mathbf{u} of the ice flow model can be obtained by minimizing the energy functional Es(u)\mathcal{E}_{\mathrm{s}}(\mathbf{u}) subject to the divergence-free constraint, the traction-free boundary condition (11), and the Dirichlet boundary conditions (12) and (13).

Non-sliding bottom boundary. With non-sliding boundary condition at the bottom, i.e., u=0\mathbf{u}=0 on Γb\Gamma_{\mathrm{b}}, we do not need to impose the boundary condition (13) using the boundary integral as the sliding boundary case in (15). Following a similar derivation as above, the velocity field u\mathbf{u} of the ice flow model with non-sliding boundary can be obtained by minimizing the energy functional

Ens(u)=∫Ω(2n1+nA−1n(12ϵu:ϵu)1+n2n−ρg⋅u) dV,(17)\mathcal{E}_{\mathrm{ns}}(\mathbf{u})=\int_{\Omega}\left(\frac{2n}{1+n}A^{-\frac{1}{n}}\left(\frac{1}{2}\boldsymbol{\epsilon}_{\mathbf{u}}:\boldsymbol{\epsilon}_{\mathbf{u}}\right)^{\frac{1+n}{2n}}-\rho\mathbf{g}\cdot\mathbf{u}\right)\,\mathrm{d}V, \tag*{(17)}

subject to the divergence-free constraint, the traction-free boundary condition (11), and the Dirichlet boundary conditions u=0\mathbf{u}=0 on Γb\Gamma_{\mathrm{b}}. The corresponding Lagrangian functional becomes

Lns(u,p)=Ens(u)−∫Ωp∇⋅u dV.(18)\mathcal{L}_{\mathrm{ns}}(\mathbf{u},p)=\mathcal{E}_{\mathrm{ns}}(\mathbf{u})-\int_{\Omega}p\nabla\cdot\mathbf{u}\,\mathrm{d}V. \tag*{(18)}

Divergence-free solutions spaces

Instead of searching for the saddle point solution of the Lagrangian functionals, we can construct divergence-free solution spaces for the velocity field u\mathbf{u} to reduce the constrained optimization problems to unconstrained optimization problems. In the two dimensional case, i.e., (x,y)∈Ω⊂R2(x,y)\in\Omega\subset\mathbb{R}^{2}, we can define a potential function ϕ:Ω↦R\phi:\Omega\mapsto\mathbb{R}, which leads to a vector function

u=[∂ϕ∂y,−∂ϕ∂x]T,\mathbf{u}=\left[\frac{\partial\phi}{\partial y},-\frac{\partial\phi}{\partial x}\right]^{\mathsf{T}},

that satisfies the divergence-free condition by construction. In the three dimensional case, i.e., (x,y,z)∈Ω⊂R3(x,y,z)\in\Omega\subset\mathbb{R}^{3}, we can define a vector function ϕ:Ω↦R3\boldsymbol{\phi}:\Omega\mapsto\mathbb{R}^{3}, so that

u=∇×ϕ\mathbf{u}=\nabla\times\boldsymbol{\phi}

satisfies the divergence-free condition. Using the divergence-free construction of the velocity field, we can directly minimize the energy functional Es(u)\mathcal{E}_{\mathrm{s}}(\mathbf{u}) and Ens(u)\mathcal{E}_{\mathrm{ns}}(\mathbf{u}) subject to appropriate boundary conditions without using the Lagrangian formulation.

Formulation of the DNN method

We first discuss the background of DNNs and then develop the DNN-based methods for solving the Non-Newtonian ice flow problems.

DNNs and its approximation property

There are two ingredients in defining a DNN. The first one is a linear map of the form T:Rn→RmT:R^n\to R^m, defined as T(x)=Ax+bT(x)=Ax+b, where A=(aij)∈Rm×nA=(a_{ij})\in R^{m\times n}, x∈Rnx\in R^n and b∈Rmb\in R^m. The second one is a nonlinear activation function σ:R→R\sigma:R\to R. Common examples of the activation function include the rectified linear unit (ReLU), σ(x)=max⁡(0,x)\sigma(x)=\max(0,x), the soft-plus function, σ(x)=log⁡(ex+1)\sigma(x)=\log(e^x+1), the sigmoid function, σ(x)=(1+e−x)−1\sigma(x)=(1+e^{-x})^{-1}, etc. The definition of activation function can be trivially extended to a nonlinear map σ:Rn→Rn\sigma:R^n\to R^n by applying the scalar-valued activation function element-wise to each of its inputs.

Using above definitions, we are able to define a continuous function F(x)F(x) by a composition of linear transforms and activation maps, i.e.,

F(x)=Tk∘σ∘Tk−1∘σ⋯∘T1∘σ∘T0(x),(19)F(x)=T^k\circ\sigma\circ T^{k-1}\circ\sigma\cdots\circ T^1\circ\sigma\circ T^0(x), \tag*{(19)}

where Ti(x)=Aix+biT^i(x)=A_i x+b_i with AiA_i and bib_i are unknown matrices and vectors to be estimated. Dimensions of AiA_i and bib_i are chosen to make (19) meaningful. Such a DNN is called a (k+1)(k+1)-layer DNN, which has kk hidden layers. Collecting all the unknown coefficients (e.g., AiA_i and bib_i) in (19) as θ∈Θ\theta\in\Theta, where θ\theta is a high-dimensional vector and Θ\Theta is the space of θ\theta. The DNN representation of a continuous function can be viewed as

F=F(x;θ).(20)F=F(x;\theta). \tag*{(20)}

We use F={F(⋅,θ)∣θ∈Θ}\mathbb{F}=\{F(\cdot,\theta)\mid\theta\in\Theta\} to denote the set of all expressible functions by the DNN parameterized by θ∈Θ\theta\in\Theta. Then F\mathbb{F} provides an efficient way to represent unknown continuous functions, comparing with a linear solution space used in classic numerical methods, e.g., a trial space spaced by linear nodal basis functions in the FEM. In the sequel, we shall discuss the approximation property of the DNN, which is relevant to the study of the expressive power of a DNN model [9, 52].

Early investigations of the approximation property of neural networks can be found in [10, 21], where the authors analyzed the approximation property for the function class given by a feed-forward neural network with a single hidden layer. Later, many authors analyzed the error estimates of such neural networks in terms of the number of neurons, the number of network layers, and the choice of activation function; see [13, 45] for a throughout review of relevant works.

In recent years, DNNs have shown successful applications in a broad range of problems, including classification of complex systems and construction of response surfaces for high-dimensional models. Significant efforts have been devoted to investigate the dependence of the expressive power of DNNs on the network configuration. For example, in [9], the authors proved that convolutional DNNs were able to express multivariate functions given in so-called hierarchical tensor formats. In [60], the author studied the expressive power of shallow and deep neural networks with piece-wise linear activation functions and established new rigorous upper and lower bounds for the network complexity in approximating Sobolev spaces. In [18], the authors proved that the continuous piece-wise bilinear finite element space is embedded in the space of DNNs constructed from the ReLU activation function, a sufficient width and a sufficient depth. Thus, we can uses DNNs to approximate the solution space spanned by the FEM basis.

Formulation of the variational neural network approach

The model problem of the ice flow with nonlinear rheology (13) can be solved by using numerical methods such as FEMs and FDMs. However, the nonlinearity in the problem bring essential difficulty in solving (13) by the traditional numerical methods. Inspired by the recent development of deep learning based numerical method for solving variational problems [12, 57], we will develop a variational DNN method to solve the glacier model problem defined in (13).

According to the divergence free property, i.e. ∇⋅u=0\nabla\cdot\mathbf{u} = 0, we can represent the velocity field u\mathbf{u} as follows:

{[∂ϕ0∂y,−∂ϕ0∂x]Tin 2D∇×(ϕ1ϕ2ϕ3)in 3D(21)\begin{cases} \left[\dfrac{\partial\phi^0}{\partial y},-\dfrac{\partial\phi^0}{\partial x}\right]^{\mathsf{T}} & \text{in 2D}\\ \nabla\times\begin{pmatrix}\phi^1\\\phi^2\\\phi^3\end{pmatrix} & \text{in 3D} \tag*{(21)} \end{cases}

where ϕi\phi^i is a scalar-valued function approximated by Fi(x,θ)F^i(\mathbf{x},\theta). Here, Fi(x,θ)F^i(\mathbf{x},\theta) is a DNN representation with x∈Rd\mathbf{x} \in\mathbb{R}^d as its input and scalar output defined in (19). Moreover, θ\theta denotes all the parameters that will be determined during the training stage. Denoting the DNN representation of the velocity field by u~\tilde{\mathbf{u}}, the numerical solution of (13) can be obtained by finding θ∈Θ\theta\in\Theta that minimizes the energy functional Es(u~)\mathcal{E}_s(\tilde{\mathbf{u}}) subject to various boundary conditions.

Ideally, we want to define u~\tilde{\mathbf{u}} using θ∈Θb⊂Θ\theta\in\Theta_b \subset\Theta, where Θb\Theta_b is the maximal parameter subset such that the resulting u~\tilde{\mathbf{u}} satisfies the traction-free boundary condition (11), and the Dirichlet boundary conditions (12) and sliding bottom boundary condition (13). After parameterizing the expressible function space by θ∈Θb\theta\in\Theta_b, we equivalently define the variational problem (15) as

min⁡θ∈ΘbJ(θ)=Es(u~)=∫Ω(2n1+nA−1n(12u~˙:u~˙)1+n2n−ρg⋅u~)dV+12∫Γbβ(Tu~)⋅(Tu~) dS.(22)\min_{\theta\in\Theta_b} J(\theta) = \mathcal{E}_s(\tilde{\mathbf{u}}) = \int_{\Omega}\left(\frac{2n}{1+n}A^{-\frac{1}{n}}\left(\frac{1}{2}\dot{\tilde{\mathbf{u}}}:\dot{\tilde{\mathbf{u}}}\right)^{\frac{1+n}{2n}}-\rho\mathbf{g}\cdot\tilde{\mathbf{u}}\right)dV+\frac{1}{2}\int_{\Gamma_b}\beta(\mathbf{T}\tilde{\mathbf{u}})\cdot(\mathbf{T}\tilde{\mathbf{u}})\,dS. \tag*{(22)}

Note that the divergence-free condition in (11) is automatically satisfied by the representation (21), and thus no additional treatment is needed.

The variational problems (22) is not convex in general and the integrals in (22) do not have a closed-form expression. Thus we numerically approximate the integrals by the Monte Carlo method and use the stochastic gradient descent (SGD) method [4] to minimize the objective function after Monte Carlo approximation. Denoting θk\theta_k the kkth component of the high-dimensional vector θ\theta, the derivative of J(θ)J(\theta) with respect to θk\theta_k can be approximated as

∂J(θ)∂θk≈vol⁡(Ω)N∑i=1N∂θk((2n1+nA−1n(12ε~:ε~)1+n2n)(xi)−ρg⋅u~(xi))+area⁡(Γb)Nb∑j=1Nb∂θk(12β(Tu)~⋅(Tu~)(yj)),(23)\frac{\partial J(\theta)}{\partial\theta_k} \approx\frac{\operatorname{vol}(\Omega)}{N} \sum_{i=1}^{N} \partial_{\theta_k} \left( \left(\frac{2n}{1+n} A^{-\frac{1}{n}} \left(\frac{1}{2}\tilde{\boldsymbol{\varepsilon}}:\tilde{\boldsymbol{\varepsilon}}\right)^{\frac{1+n}{2n}}\right)(x_i)-\rho\boldsymbol{g}\cdot\tilde{\boldsymbol{u}}(x_i) \right) + \frac{\operatorname{area}(\Gamma_b)}{N_b}\sum_{j=1}^{N_b}\partial_{\theta_k}\left(\frac{1}{2}\beta(\mathbf{T}\tilde{\boldsymbol{u})}\cdot(\mathbf{T}\tilde{\boldsymbol{u}})(y_j)\right), \tag*{(23)}

where random samples xi∼i.i.d.Unif⁡(Ω)x_i \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Unif}(\Omega) are uniformed drawn from the physical domain Ω\Omega, random samples yj∼i.i.d.Unif⁡(Γb)y_j \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Unif}(\Gamma_b) are uniformed drawn from the bottom boundary Γb\Gamma_b, vol⁡(Ω)\operatorname{vol}(\Omega) is the volume of the domain and area⁡(Γb)\operatorname{area}(\Gamma_b) is the area of bottom boundary. In the context of deep learning method, NN and NbN_b are called batch numbers, which mean the number of training examples utilized in one iteration.

After approximating the gradient of J(θ)J(\theta), we can update each component of θ\theta as follows:

θkn+1=θkn−η∂J(θ)∂θk∣θk=θkn,(24)\theta_k^{n+1}=\theta_k^n-\eta\left.\frac{\partial J(\theta)}{\partial\theta_k}\right|_{\theta_k=\theta_k^n}, \tag*{(24)}

where η\eta is the learning rate. To accelerate the training of the neural network, we use the Adam version of the SGD method [27].

In the objective function (26), it is rather challenging to restrict the neural network parameter θ\theta to the subset Θb\Theta_b that satisfies the boundary conditions, because the boundary of the subset may have complicated geometry. To address this issue, we adopt a relaxation approach by imposing boundary conditions as penalty terms. For each of the boundary conditions in (11)-(13), we define a linear map BjB_j that maps the solution u~\tilde{\boldsymbol{u}} to the residual of the boundary constraint, which defines a boundary integral

Bj(θ)=∫∂Dj∥Bju~(x,θ)∥2 dxB_j(\theta)=\int_{\partial D_j}\lVert B_j\tilde{\boldsymbol{u}}(x,\theta)\rVert^2\,\mathrm{d}x

for the boundary ∂Dj\partial D_j. Then, the boundary conditions can be imposed as soft constraints via penalty terms to the objective functional J(⋅)J(\cdot) in (26). This leads to a new objective functional

u~ε=arg⁡min⁡θ∈Θ(J(θ)+∑j=1b1εjBj(θ)),(25)\tilde{u}_{\varepsilon}=\mathop{\arg\min}_{\theta\in\Theta}\left(J(\theta)+\sum_{j=1}^{b}\frac{1}{\varepsilon_j}B_j(\theta)\right), \tag*{(25)}

where bb is the number of boundary conditions needs to be imposed. Note that when a penalty term εj−1Bj(θ)\varepsilon_j^{-1}B_j(\theta) approaches zero as εj→0\varepsilon_j\to0, the corresponding boundary condition is satisfied in the weak sense.

Implementation details

In addition to the conventional DNN formulation for solving PDE, we also made some modification that are shown to be essential in resolving the scale of the problem domain and the scale of physical parameters in real world problems.

Normalizing Layer. In some glacier model problems, periodic boundary conditions are used. For example, periodic boundary conditions are imposed in the two horizontal directions in the in the 2D model presented in Example C of the benchmarks [42]. To this end, we add an extra layer between input and first dense layer of our network. That maps (x,y)(x,y) to terms like (cos⁡(2πx/T),sin⁡(2πx/T),cos⁡(2πy/T),sin⁡(2πy/T))(\cos(2\pi x/T),\sin(2\pi x/T),\cos(2\pi y/T),\sin(2\pi y/T)), where TT is the period in the horizontal dimensions.

In most of the glacier modelling problems, the vertical scaling can be two or three orders of magnitude smaller than the horizontal scaling. To resolve this scaling, we also introduce a reparametrization: x↦x/Lx \mapsto x/L, where LL is the diameter of the domain in corresponding dimension, to the normalization layer from input. Resolving this scaling is essential to ensure the expressibility of neural networks for the problems we are targeting here, because nonlinearity can be represented by a neural network may only exist in a bounded input domain. Taking a scalar single layer perception with the Sigmoid activation function σ(z)=(1+e−z)−1\sigma(z)=(1+e^{-z})^{-1} as an example, we have

f(x)=W2σ(W1x+b1)+b2.f(x)=W_2\sigma(W_1x+b_1)+b_2.

When W1x+b1W_1x+b_1 is sufficiently large (e.g., W1x+b1≫5W_1x+b_1\gg5), f(x)f(x) becomes a constant approximately. Thus, to sufficiently express nonlinear functions, scale of W1W_1 has to be O(L−1)\mathcal{O}(L^{-1}) without any reparametrization. However, most of existing implementations of machine learning toolboxes only support single-precision float-point numbers. This leads to inaccurate representation of W1W_1 and the gradient of the objective function, as the length scale of most of the real-world problems can be O(104)−O(106)\mathcal{O}(10^4)-\mathcal{O}(10^6). The reparametrization is critical to ensure the expressibility of neural networks and the numerical stability in these real-world problems.

Balancing penalty parameters of boundary conditions. Except for periodic boundary conditions, as shown in (25), the other boundary conditions are enforced by penalty terms. However, there is no general framework to choose the penalty weight εj\varepsilon_j in the context of DNN. It may not be suitable to use the same penalty parameter εj\varepsilon_j for different boundary conditions, as the magnitude of the boundary maps Bj(θ)B_j(\theta) can vary significantly. For example, in the 3D model presented in § 4.3, the term Bj(θ)B_j(\theta) is O(10−4)\mathcal{O}(10^{-4}) for the traction-free condition at the top boundary and O(106)\mathcal{O}(10^6) for the basal friction condition at the bottom boundary. Thus, finding the correct penalty parameter εj\varepsilon_j for each of the boundary conditions is critical in balancing the soft constraints due to various boundary conditions. We propose to first estimate the scaling relations between each of the boundary integrals Bj(⋅)B_j(\cdot) with the objective functional J(⋅)J(\cdot) at the initialization stage of the network training. Denoting the initial parameter by θ0\theta_0, we can then assign εj\varepsilon_j as

εj=1ε0Bj(θ0)J(θ0),\varepsilon_j=\frac{1}{\varepsilon_0}\frac{B_j(\theta_0)}{J(\theta_0)},

so that the objective functional J(⋅)J(\cdot) and each of the boundary integrals Bj(⋅)B_j(\cdot) can be approximately balanced during the training. Here the common factor ε0\varepsilon_0 is chosen to be 50, which is an empirical constant shown to be sufficient in previous research [24] and in our numerical experiments on simpler nonlinear models.

Network Hyperparameters. In the numerical experiments of this work, we will apply the same network to represent ϕ\phi in both two and three dimensions. After the normalizing layer, it has six latent dense layers with width 10. Between these dense layers we apply the Sigmoid activation function to guarantee the smoothness of our representation. There is no activation function between the second last layer and the output layer. We illustrate the structure of our network in Figure 1.

The network layout of $\phi$

Figure 1. The network Layout of ϕ\phi.

Numerical Results

In this section, we shall present numerical results in solving non-Newtonian Stokes equations to demonstrate the performance of the propose method. First we consider a two dimensional synthetic model with an analytical solution to demonstrate the efficiency and accuracy of our method. Then we apply the developed method on realistic benchmark problems [42] to demonstrate its generality. This includes a two dimensional model of the Arolla glacier and a three dimensional box model.

A 2D Model on irregular domain

We start with a two dimensional model defined on a domain that is enclosed by

y=0andy=12x(1−x)withx∈[0,1].y = 0 \quad\text{and} \quad y = \frac{1}{2}x(1-x) \quad\text{with} \quad x \in[0,1].

To setup the benchmark, we start with a ground truth potential function

ϕ=exp⁡(x)(x−2)2y(y−1)2,\phi= \exp(x)(x-2)^2y(y-1)^2,

which leads to an analytical expression of the velocity field

{u=exp⁡(x)(x−2)2(1−4y+3y2)v=−exp⁡(x)(x−2)2y(y−1)2\begin{cases} u = \exp(x)(x-2)^2(1-4y+3y^2) \\ v = -\exp(x)(x-2)^2y(y-1)^2 \end{cases}

The above velocity field satisfies the conservation of mass condition in Eq. (10) and the no-penetration boundary condition in Eq. (12) at the bottom boundary Γb\Gamma_b. Then, by substituting this velocity field into Eq. (9), we obtain an explicit expression of the forcing term f\mathbf{f} in Eq. (9), a function p0p_0 for the traction-free boundary condition Eq. (11) at the top boundary Γt\Gamma_t, and a function β\beta for the sliding boundary condition Eq. (13) at the bottom boundary Γb\Gamma_b.

Since the shape of the domain is a semicircle in this example, it is not necessary to use normalization layers or balance regularization factors. We use a constant regularization factor 1ϵ=50\frac{1}{\epsilon}=50 for all boundary conditions. During training, in every step of stochastic gradient descent, we use 5000 random samples uniformly distributed in the domain Ω\Omega to evaluate Eq. (23) and 1000 uniform random samples on the boundary Γ\Gamma to construct the soft constraints in Eq. (11)–Eq. (13). The learning rate, which is the step length of SGD, is set to be 10−310^{-3}. We renew the training data every 200 steps of learning. The configuration of the neural network is illustrated in Figure 1.

In Figure 2, we compare the results of our proposed method with the ground truth. Since the velocity field is invariant to the potential function ϕ\phi up to a constant shift, we shift the output of ϕ\phi learned by neural networks to ensure it has the same average as the ground truth in the comparison. For both of the velocity field and the potential function, the neural network offers comparable results with the ground truth.

Solution of synthetic model, showing panels (a) $u$, (b) $v$, (c) $\phi$ rescaled, (d) ref: $u$, (e) ref $v$, and (f) ref $\phi$

Figure 2. Solution of synthetic model

In Figure 3 we compare the relative L2L_2 error of the velocity field with the energy functional Es(u)\mathcal{E}_s(\mathbf{u}) in Eq. (15). We observe that Es(u)\mathcal{E}_s(\mathbf{u}) follows a similar trend to the relative L2L_2 error. The relative L2L_2 error drops to 0.005 after 10000 steps of training.

Figure 3. Training loss (Lagrangian) and validation error (L2L_2 relative error) with respect to training steps

Surface velocity along surface direction, and $u$ and $v$ fields

Figure 5. (a) Surface Velocity along surface direction; (b) uu; (c) vv. Noemi’s paper

The Arolla model

The two-dimensional model of the Arolla glacier is a diagnostic experiment along the central flow-line of a temperate glacier in the European Alps (Haut Glacier d’Arolla, Switzerland), based on earlier experiments by [3, 41]. The domain of this model is enclosed by the longitudinal surface and bedrock profiles of Haut Glacier d’Arolla (as shown in Figure 4 (b) and (c)). This model has been used as a benchmark example in various investigations of the forward modelling and the inverse modelling of glaciers, see [43] for some notable examples.

Surface velocity along surface direction, and $u$ and $v$ fields

Figure 4. (a) Surface Velocity along surface direction; (b) uu; (c) vv. Arolla

We consider two experiments in this example. The first experiment follows the setup of [42 Section 3.5], in which a no-sliding bottom boundary with β=∞\beta= \infty is considered. In the second experiment, we use a sliding bottom boundary with the function

exp⁡(β)={1000+1000sin⁡(2πx5000)0≤x<37501000(16−x250)3750≤x<400010004000≤x<5000\exp(\beta) = \begin{cases} 1000 + 1000 \sin\left(\frac{2\pi x}{5000}\right) & 0 \le x < 3750 \\ 1000\left(16 - \frac{x}{250}\right) & 3750 \le x < 4000 \\ 1000 & 4000 \le x < 5000 \end{cases}

which is the same as the work of [43 Section 4.2].

A 3D box model

We also consider the three dimensional box model presented in [42 Section 3]. In this model, a slab of ice sliding down a sloping bed with a constant incline angle α=0.1\alpha= 0.1 degree is considered. The ice slab is enclosed in a box-shaped domain [0,L]×[0,L]×[0,H][0,L] \times[0,L] \times[0,H], where L=20×103L = 20 \times10^{3} metres, and H=103H = 10^{3} meters. After a change of coordinate that align the xx-axis with the sliding surface (see 6(a)), the vector ρg\boldsymbol{\rho g} is given as

ρg=910×9.81×[sin⁡θ,0,−cos⁡θ] Pa,\boldsymbol{\rho g} = 910 \times9.81 \times[\sin\theta, 0, -\cos\theta]\ \mathrm{Pa},

where θ=π/1800\theta= \pi/1800. In this example, periodic boundary conditions are applied to boundaries in the horizontal directions, the traction-free boundary conditions in Eq. (11) is applied to the top boundary, and the sliding boundary condition in Eq. (12) and Eq. (13) is prescribed at the bottom boundary. The basal sliding coefficient is defined as

β(x,y)=1000+1000sin⁡(2πxL)sin⁡(2πyL).\beta(x,y) = 1000 + 1000 \sin\left(\frac{2\pi x}{L}\right) \sin\left(\frac{2\pi y}{L}\right).

In Figure 6(b) we plot the value of basal friction coefficient β\beta. Empirically speaking, the surface speed becomes larger when β\beta is small. The ground truth solution is unknown. In Figure 7 we plot the surface velocity against yy direction at the slice x=L4x = \frac{L}{4}. In addition we compare our prediction with various result in [42] (experiment C, L=20kmL = 20\mathrm{km}).

Setting of box model, showing coordinate change and basal coefficient contours

Figure 6. Setting of box Model

Figure 7. Surface velocity, Benchmark paper, experiment C, 20km. Solid line: Our prediction; Dashed lines: various prediction in Benchmark paper

Conclusions

In this paper, we investigate deep-learning methods to simulate non-Newtonian ice flow models. By formulating the model problem into a variational form and constructing a divergence-free solution space using DNN, we convert the solution of the nonlinear PDE into the solution of an optimization problem. Despite the rather high-dimensionality of the parameter space of the DNN, the solution can still be efficiently obtained by the SGD method. In this framework, once we are able to sample from the problem domain and its boundary, no special treatment is needed to handle irregular boundaries of the problem domain. Therefore, the proposed method is easy to implement and mesh-free. To address the application in real-world computation, we introduce normalizing layers and adaptive boundary penalties in the configuration of our network. Finally, we present numerical experiments to demonstrate the performance of the proposed method. Specifically, we use the DNN method to solve ice flow model for the 2D Arrola Glacier and a 3D box model with real-world scaling. Our numerical experiments demonstrate that the DNN method provides satisfactory simulation results for glacier flows. We expect this method can be applied to a more general class of problems in non-Newtonian mechanics that satisfies certain variational principles.

Acknowledgements

The research of T. Cui is supported by the Australian Research Council under the grant DP21010309. The research of Z. Zhang is supported by Hong Kong RGC grant (Projects 17300318 and 17307921), National Natural Science Foundation of China (Project 12171406), Seed Funding Programme for Basic Research (HKU) and Seed Funding for Strategic Interdisciplinary Research Scheme 2021/22 (HKU).

References

  1. [1]G. Bao, X. Ye, Y. Zhang, and H. Zhou, Numerical solution of inverse problems by weak adversarial networks, Inverse Problems, 36(11) (2020), p. 115003.DOI
  2. [2]Y. Bar-Sinai, S. Hoyer, J. Hickey, and M. Brenner, Learning data-driven discretizations of PDEs, Bulletin of the American Physical Society, 63 (2018).
  3. [3]H. Blatter, G. K. Clarke, and J. Colinge, Stress and velocity fields in glaciers: Part ii. sliding and basal stress distribution, Journal of Glaciology, 44 (1998), pp. 457–466.DOI
  4. [4]L. Bottou, Large-scale machine learning with stochastic gradient descent, in Proceedings of COMPSTAT’2010, Springer, 2010, pp. 177–186.
  5. [5]J. Brown, Efficient nonlinear solvers for nodal high-order finite elements in 3d, Journal of Scientific Computing, 45 (2010), pp. 48–63.DOI
  6. [6]J. Brown, B. Smith, and A. Ahmadia, Achieving textbook multigrid efficiency for hydrostatic ice sheet flow, SIAM Journal on Scientific Computing, 35 (2013), pp. B359–B375.DOI
  7. [7]Z. Cai, J. Chen, and M. Liu, Least-squares ReLU neural network (LSNN) method for linear advection-reaction equation, Journal of Computational Physics, (2021), p. 110514.DOI
  8. [8]Z. Cai, J. Chen, M. Liu, and X. Liu, Deep least-squares methods: An unsupervised learning-based numerical method for solving elliptic PDEs, Journal of Computational Physics, 420 (2020), p. 109707.DOI
  9. [9]N. Cohen, O. Sharir, and A. Shashua, On the expressive power of deep learning: A tensor analysis, in Conference on Learning Theory, 2016, pp. 698–728.arxiv.org/abs/1509.05009
  10. [10]G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of control, signals and systems, 2 (1989), pp. 303–314.
  11. [11]J. K. Dukowicz, S. F. Price, and W. H. Lipscomb, Consistent approximations and boundary conditions for ice-sheet dynamics from a principle of least action, Journal of Glaciology, 56 (2010), pp. 480–496.DOI
  12. [12]W. E and B. Yu, The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics, 6 (2018), pp. 1–12.arxiv.org/abs/1710.00211
  13. [13]S. Ellacott, Aspects of the numerical analysis of neural networks, Acta Numerica, 3 (1994), pp. 145–202.DOI
  14. [14]H. C. Elman, D. J. Silvester, and A. J. Wathen, Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics, Numerical Mathematics and Scie, 2014.
  15. [15]M. R. Fraters, W. Bangerth, C. Thieulot, A. Glerum, and W. Spakman, Efficient and practical newton solvers for non-linear stokes systems in geodynamic problems, Geophysical Journal International, 218 (2019), pp. 873–894.DOI
  16. [16]I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning, vol. 1, MIT press Cambridge, 2016.
  17. [17]J. Han, A. Jentzen, and W. E, Solving high-dimensional partial differential equations using deep learning, Proceedings of the National Academy of Sciences, 115 (2018), pp. 8505–8510.DOI
  18. [18]J. He, L. Li, J. Xu, and C. Zheng, Relu deep neural networks and linear finite elements, Journal of Computational Mathematics, 38 (2020), pp. 502–527.DOI
  19. [19]J. He and J. Xu, Mgnet: A unified framework of multigrid and convolutional neural network, Science China Mathematics, 62 (2019), pp. 1331–1354.DOI
  20. [20]V. Heuveline and F. Schieweck, On the inf-sup condition for higher order mixed fem on meshes with hanging nodes, ESAIM: Mathematical Modelling and Numerical Analysis, 41 (2007), pp. 1–20.DOI
  21. [21]K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural networks, 2 (1989), pp. 359–366.DOI
  22. [22]K. Hutter, Theoretical glaciology: material science of ice and the mechanics of glaciers and ice sheets, vol. 1, Springer, 2017.
  23. [23]T. Isaac, G. Stadler, and O. Ghattas, Solution of nonlinear stokes equations discretized by high-order finite elements on nonconforming and anisotropic meshes, with application to ice sheet dynamics, SIAM Journal on Scientific Computing, 37 (2015), pp. B804–B833.DOI
  24. [24]Jingrun, Chen, , 8405, , J. Chen, Rui, Du, , 8406, , R. Du, Keke, Wu, , 8407, , and K. Wu, A comparison study of deep galerkin method and deep ritz method for elliptic problems with different boundary conditions, Communications in Mathematical Research, 36 (2020), pp. 354–376.DOI
  25. [25]S. Karumuri, R. Tripathy, I. Bilionis, and J. Panchal, Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks, Journal of Computational Physics, 404 (2020), p. 109120.DOI
  26. [26]Y. Khoo, J. Lu, and L. Ying, Solving parametric PDE problems with artificial neural networks, European Journal of Applied Mathematics, Special Issue 3: Connections between Deep Learning and Partial Differential Equations, 32 (2021), pp. 421–435.DOI
  27. [27]D. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980, (2014).arxiv.org/abs/1412.6980
  28. [28]I. Lagaris, A. Likas, and D. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9 (1998), pp. 987–1000.DOI
  29. [29]Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, 521 (2015), p. 436.DOI
  30. [30]H. Lee and I. S. Kang, Neural algorithm for solving differential equations, Journal of Computational Physics, 91 (1990), pp. 110–131.DOI
  31. [31]Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv preprint arXiv:2010.08895, (2020).arxiv.org/abs/2010.08895
  32. [32]Z. Long, Y. Lu, and B. Dong, PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, Journal of Computational Physics, 399 (2019), p. 108925.DOI
  33. [33]Z. Long, Y. Lu, X. Ma, and B. Dong, PDE-Net: Learning PDEs from data, International Conference on Machine Learning, (2018), pp. 3208–3216.arxiv.org/abs/1710.09668
  34. [34]J. Lu, Z. Shen, H. Yang, and S. Zhang, Deep network approximation for smooth functions, SIAM Journal on Mathematical Analysis, 53 (2021), pp. 5465–5506.DOI
  35. [35]L. Lu, P. Jin, and G. Karniadakis, Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators, arXiv:1910.03193, (2019).DOI
  36. [36]D. A. May, J. Brown, and L. Le Pourhiet, A scalable, matrix-free multigrid preconditioner for finite element discretizations of heterogeneous stokes flow, Computer methods in applied mechanics and engineering, 290 (2015), pp. 496–523.DOI
  37. [37]D. McKenzie, The generation and compaction of partially molten rock, Journal of petrology, 25 (1984), pp. 713–765.
  38. [38]A. Meade and A. Fernandez, The numerical solution of linear ordinary differential equations by feedforward neural networks, Math. Comput. Model., 19 (1994), pp. 1–25.DOI
  39. [39]H. Montanelli and Q. Du, New error bounds for deep ReLU networks using sparse grids, SIAM Journal on Mathematics of Data Science, 1 (2019), pp. 78–92.arxiv.org/abs/1712.08688
  40. [40]W. S. B. Paterson, Physics of glaciers, Butterworth-Heinemann, 1994.
  41. [41]F. Pattyn, Transient glacier response with a higher-order numerical ice-flow model, Journal of Glaciology, 48 (2002), pp. 467–477.DOI
  42. [42]F. Pattyn, L. Perichon, A. Aschwanden, B. Breuer, B. De Smedt, O. Gagliardini, G. H. Gudmundsson, R. C. Hindmarsh, A. Hubbard, J. V. Johnson, et al., Benchmark experiments for higher-order and full-stokes ice sheet models (ismip-hom), The Cryosphere, 2 (2008), pp. 95–108.DOI
  43. [43]N. Petra, J. Martin, G. Stadler, and O. Ghattas, A computational framework for infinite-dimensional bayesian inverse problems, part ii: Stochastic newton mcmc with application to ice sheet flow inverse problems, SIAM Journal on Scientific Computing, 36 (2014), pp. A1525–A1555.arxiv.org/abs/1308.6221
  44. [44]N. Petra, H. Zhu, G. Stadler, T. J. Hughes, and O. Ghattas, An inexact gauss-newton method for inversion of basal sliding and rheology parameters in a nonlinear stokes ice sheet model, Journal of Glaciology, 58 (2012), pp. 889–903.
  45. [45]A. Pinkus, Approximation theory of the MLP model in neural networks, Acta numerica, 8 (1999), pp. 143–195.DOI
  46. [46]T. Qin, K. Wu, and D. Xiu, Data driven governing equations approximation using deep neural networks, Journal of Computational Physics, 395 (2019), pp. 620–635.DOI
  47. [47]M. Raissi, P. Perdikaris, and G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics, 378 (2019), pp. 686–707.DOI
  48. [48]J. Rudi, A. C. I. Malossi, T. Isaac, G. Stadler, M. Gurnis, P. W. Staar, Y. Ineichen, C. Bekas, A. Curioni, and O. Ghattas, An extreme-scale implicit solver for complex pdes: highly heterogeneous flow in earth’s mantle, in Proceedings of the international conference for high performance computing, networking, storage and analysis, 2015, pp. 1–12.DOI
  49. [49]S. Rudy, J. Kutz, and S. Brunton, Deep learning of dynamics and signal-noise decomposition with time-stepping constraints, Journal of Computational Physics, 396 (2019), pp. 483–506.DOI
  50. [50]G. Schubert, D. L. Turcotte, and P. Olson, Mantle convection in the Earth and planets, Cambridge University Press, 2001.DOI
  51. [51]C. Schwab and M. Suri, Mixed hp finite element methods for stokes and non-newtonian flow, Computer methods in applied mechanics and engineering, 175 (1999), pp. 217–241.DOI
  52. [52]C. Schwab and J. Zech, Deep Learning in High Dimension, Research Report, vol. 2017 (2017).
  53. [53]Z. Shen, H. Yang, and S. Zhang, Deep network with approximation error being reciprocal of width to power of square root of depth, Neural Computation, 33 (2021), pp. 1005–1036.DOI
  54. [54]J. Sirignano and K. Spiliopoulos, DGM: A deep learning algorithm for solving partial differential equations, Journal of Computational Physics, 375 (2018), pp. 1339–1364.DOI
  55. [55]R. Stenberg and M. Suri, Mixed hp finite element methods for problems in elasticity and stokes flow, Numerische Mathematik, 72 (1996), pp. 367–389.DOI
  56. [56]Y. Wang, S. Cheung, E. Chung, Y. Efendiev, and M. Wang, Deep multiscale model learning, Journal of Computational Physics, 406 (2020), p. 109071.arxiv.org/abs/1806.04830
  57. [57]Z. Wang and Z. Zhang, A mesh-free method for interface problems using the deep learning approach, Journal of Computational Physics, 400 (2020), p. 108963.DOI
  58. [58]K. Wu and D. Xiu, Data-driven deep learning of partial differential equations in modal space, Journal of Computational Physics, (2020), p. 109307.DOI
  59. [59]L. Yang, X. Meng, and G. Karniadakis, B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data, J. Comp. Physics, 425 (2021), p. 109913.DOI
  60. [60]D. Yarotsky, Error bounds for approximations with deep ReLU networks, Neural Networks, 94 (2017), pp. 103–114.DOI
  61. [61]Y. Zang, G. Bao, X. Ye, and H. Zhou, Weak adversarial networks for high-dimensional partial differential equations, Journal of Computational Physics, 411 (2020), p. 109409.DOI
  62. [62]Y. Zhu and N. Zabaras, Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, Journal of Computational Physics, 366 (2018), pp. 415–447.DOI
  63. [63]Y. ZHU, N. ZABARAS, P. KOUTSOURELAKIS, AND P. PERDIKARIS, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, Journal of Computational Physics, 394 (2019), pp. 56–81.DOI

Paper details

vlnc:W_8rc7yatm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents