betaFeedback

WorksComputer Science

A Memory Efficient Adjoint Method to Enable Billion Parameter Optimization on a Single GPU in Dynamic Problems

Abstract

Dynamic optimization is currently limited by sensitivity computations that require information from full forward and adjoint wave fields. Since the forward and adjoint solutions are computed in opposing time directions, the forward solution must be stored. This requires a substantial amount of memory for large-scale problems even when using check pointing or data compression techniques. As a result, the problem size is memory bound rather than bound by wall clock time, when working with modern GPU-based implementations that have limited memory capacity. To overcome this limitation, we introduce a new approach for approximate sensitivity computation based on the adjoint method (for self-adjoint problems) that relies on the principle of superposition. The approximation allows an iterative computation of the sensitivity, reducing the memory burden to that of the solution at a small number of time steps, i.e., to the number of degrees of freedom. This enables sensitivity computations for problems with billions of degrees of freedom on current GPUs, such as the A100 from NVIDIA (from 2020). We demonstrate the approach on full waveform inversion and transient acoustic topology optimization problems, relying on a highly efficient finite difference forward solver implemented in CUDA. Phenomena such as damping cannot be considered, as the approximation technique is limited to self-adjoint problems.

Introduction

The key difference between graphics processing units (GPUs) and central processing units (CPUs), the latter of which are more commonly used in computational mechanics, stems from the number and design of their cores. GPUs have far more cores (≈103\approx10^{3} versus ≈100−101\approx10^{0} - 10^{1}), but each core is less sophisticated. This makes GPUs well-suited for highly parallel tasks, provided these tasks are repetitive and relatively simple (regular workloads and predictable memory accesses). In the context of the finite element method (FEM) or finite difference method (FDM), this can lead to superior performance on GPUs — especially for problems with low complexity, e.g., FDM on uniform grids. By contrast, adaptive methods, irregular meshes, and sparse computations are more challenging to implement efficiently on GPU architectures. A further limitation of GPUs is their memory capacity (≈101\approx10^{1} GB versus ≈102\approx10^{2} GB for CPUs), which can restrict their applicability to

Finite Difference Stencils as Convolutional Kernels

smaller problems.

Thus, GPUs have primarily been applied to small- to mid-sized problems in the literature. Examples hereof are [1, 2, 3] for FDM [4, 5] for FEM, [6] for the spectral element method (SEM), and [2, 7] for the finite volume method (FVM). Further attempts have been made using deep learning [8, 9] and their corresponding frameworks [10, 11, 12] with the goal to exploit advances in software and hardware from the machine learning community [13, 14]. Examples2 are available for both FEM [15, 16, 17, 18, 19] and FDM [20, 21, 22, 23].

Although GPUs have been successfully applied to forward problems, their use for inverse optimization problems is still limited. A notable exception in static elasticity is [5]. Unlike in static elasticity, the memory limitation in dynamic optimization problems originates from the sensitivity computation. Computing the sensitivity with the adjoint method [24, 25, 26, 27] requires information about both the forward and adjoint wave fields at the same time. Since the forward wave field is solved forward in time, while the adjoint wave field is solved backward in time, the forward wave field must be saved. Storing the full forward solution requires enormous amounts of memory, as it involves the number of degrees of freedom multiplied by the number of time steps. Even worse, the excitation frequency ω\omega dictates the spatial and temporal resolutions. For a dd-dimensional problem, the size of the full wave field scales with ∝ωd+1\propto\omega^{d+1}, i.e., doubling the frequency for a three dimensional problem increases the memory requirement by a factor of 16. This high memory demand is difficult to satisfy with current GPU memory capacities, and imposes a restrictive bound upon the size of computable problems. State-of-the-art remedies are checkpointing [28, 29, 30, 31], data compression [32, 33], and multi-GPU implementations [1, 3, 34, 35, 36]. Checkpointing stores only selective forward states. Since the remaining states need to be recomputed during the adjoint computation, checkpointing leads to an increase in the computational effort. Data compression reduces the resolution of the stored forward solution, which may result in a degradation of the accuracy of the sensitivity. Finally, multi-GPU usage requires additional (and expensive) hardware with greater communication overheads. In [37, 38], the authors re-simulate the forward wavefield along the adjoint wavefield backward in time, allowing memory-efficient gradient computations. Additionally, they use a random boundary approach to implement absorbing boundary conditions.

We propose an alternative workaround that does not increase the computational effort. This approach relies on the superposition of forward and adjoint wave fields. Corresponding sensitivities are computed approximately with the adjoint method in an almost memory-free manner. Only information of current time steps are required. This reduces the memory requirement from the full forward wave solution to the forward solution at a small number of time steps. Given 40 GB of memory, available in a single A100 GPU chip, renders it possible to compute sensitivities of problems with billions of degrees of freedom. As the proposed approach relies on time reversibility, its application is limited to self-adjoint problems and time-reverse time integration schemes. We study the efficiency and accuracy of the proposed approach on two inverse problems based on the wave equation: full waveform inversion (FWI) [39, 40] and transient acoustic topology optimization (TATO) [41, 42, 43, 44, 45, 46].3 In both cases, a scalar wave equation without damping is considered. The equations are solved with a highly efficient, yet simple, finite difference solver, implemented in CUDA® [50]. The code is available at [51]. Due to its simplicity, the code can readily be adapted to other dynamic forward problems and optimization scenarios.

Before explaining the approximate memory-efficient sensitivity computation in § 5, we introduce FWI and TATO in § 3, and give details on the GPU-based finite difference implementation in § 4. In § 6, we demonstrate the efficiency and robustness of the introduced method on various optimization problems. Finally, § 7 concludes our findings.

Dynamic Optimization Problems

Underlying Problem

In FWI and TATO, we aim to minimize a cost functional CC that provides a measure of how close a particular model is to a measured (FWI) or desired (TATO) state. In the sequel, γ\gamma is a discrete realization of the corresponding continuous field that represents spatially distributed but time invariant material parameters. The minimization of CC is carried out with respect to the variables γ\gamma,

γ∗=arg⁡min⁡γ C(γ),(1)\gamma^{*} = \underset{\gamma}{\arg\min}\ C(\gamma), \tag*{(1)}

where γ∗\gamma^{*} minimizes the cost function CC. The cost function CC is minimized in an iterative procedure using a gradient based approach. To this end, a sensitivity analysis dC/dγ=∇γC\mathrm{d}C/\mathrm{d}\gamma= \nabla_{\gamma}C yields the direction of the steepest ascent. Thus, an update in the negative direction

γ(j+1)=γ(j)−αdC(u(γ(j));γ(j))dγ(j)(2)\gamma^{(j+1)} = \gamma^{(j)} - \alpha\frac{\mathrm{d}C\left(u(\gamma^{(j)});\gamma^{(j)}\right)}{\mathrm{d}\gamma^{(j)}} \tag*{(2)}

with the step length α\alpha intends to reduce the objective function CC.4 The procedure is repeated iteratively until convergence is reached. The detailed problem formulations, including governing equations, objective function formulation, and sensitivity analysis with the adjoint method, are provided in § 3.2 for FWI and in § 3.4 for TATO.

Full Waveform Inversion

FWI in a nutshell

Originally developed for its use in geophysics, FWI can also be applied to ultrasonic testing. The aim is to infer the internal state γ\gamma of a sample by measuring wave signals uu at sensor locations xix_i. FWI exploits that waves provide information about the media they traverse. Measured wave signals are the image of a defined signal injected into the sample by a source at a defined position. These wave signals, thus, carry information about possible local material anomalies they traversed on their way to the receiver positions which allows to detect local perturbations in the material. In this work we investigate the identification of voids using density-scaling FWI [56]. Figure 1 provides a conceptual illustration of FWI. The source f(x,t)f(x,t) (red circle) excites a wave uu at a position xx. After interacting with the domain Ω\Omega and the void ΩV\Omega_V, the wave uu is recorded at the sensor positions (blue circles).

Image not extracted

Figure 1. Ultrasonic testing on a domain Ω\Omega (L1×L2×L3L_1 \times L_2 \times L_3) with a void ΩV⊂Ω\Omega_V \subset\Omega. The boundaries are defined as Γ=⋃i=16Γi\Gamma= \bigcup_{i=1}^{6} \Gamma_i. The signal emitted by the source (red circle) is recorded at the sensors (blue circles) over time TT. Modified from [23].

Physical Model

In solid structures, wave propagation phenomena can be modeled by the elastic wave equation. However, the scalar wave equation can be used to investigate FWI for method development [57, 58] (see the Appendix of [23] for a brief but sufficient explanation). Thus, for the sake of simplicity, we consider the scalar wave equation. Furthermore, we consider ρ\rho-scaling to parameterize the scalar wave equation to detect void-like defects such as cavities or cracks [56]. The wave field u:Ω→Ru:\Omega\to\mathbb{R} fulfills the scalar wave equation in the domain Ω⊂Rd\Omega\subset\mathbb{R}^{d} with dimension dd for time T⊂R\mathcal{T} \subset\mathbb{R} excited by a volumetric force ff:

γ(x)ρ0u¨(x,t)−∇⋅(γ(x)ρ0c02∇u(x,t))=f(x,t)on Ω×T,(3)\gamma(x)\rho_{0}\ddot{u}(x,t) - \nabla\cdot\left(\gamma(x)\rho_{0}c_{0}^{2}\nabla u(x,t)\right) = f(x,t) \qquad\text{on }\Omega\times\mathcal{T}, \tag*{(3)}

where ρ0\rho_{0} and c0c_{0} are the density and wave speed of the undamaged material also called background material. Thus, ρ\rho-scaling can be interpreted as scaling the background density ρ0\rho_{0} with a spatially distributed scaling function γ∈[ϵ,1]\gamma\in[\epsilon,1], where 0<ϵ≪10 < \epsilon\ll1 is a non-zero lower bound that ensures numerical stability of the simulations. In this work, we set ϵ=10−5\epsilon= 10^{-5}. The material parameters used in this work are ρ0=2700 kg/m3\rho_0 = 2700\ \mathrm{kg}/\mathrm{m}^3, and c0=6000 m/sc_0 = 6000\ \mathrm{m}/\mathrm{s}.

We consider free reflecting boundaries, modeled by homogeneous Neumann boundary conditions

n⋅∇u=0on Γ,(4)\boldsymbol{n} \cdot\nabla u = 0 \qquad\text{on } \Gamma, \tag*{(4)}

where n\boldsymbol{n} is the unit normal vector on the boundary Γ\Gamma of the domain. In addition, homogeneous initial conditions are considered.

u(x,0)=u˙(x,0)=0on Ω(5)u(\boldsymbol{x},0) = \dot{u}(\boldsymbol{x},0) = 0 \qquad\text{on } \Omega \tag*{(5)}

The domain is excited by a point source at position xs\boldsymbol{x}_s which is defined using the Kronecker delta function δ\delta

f(x,t)=ψ(t) δ(x−xs).(6)f(\boldsymbol{x},t) = \psi(t)\,\delta(\boldsymbol{x}-\boldsymbol{x}_s). \tag*{(6)}

In the sequel, the time signal ψ\psi, is modeled by the sine burst with ncn_c cycles, a frequency ω\omega, and the amplitude ψ0\psi_0 such that

ψ(t)={ψ0sin⁡(ωt)sin⁡2(ωt2nc)for 0≤t≤2πncω,0for 2πncω<t.(7)\psi(t) = \begin{cases} \psi_0 \sin(\omega t)\sin^2\left(\dfrac{\omega t}{2n_c}\right) & \text{for } 0 \le t \le\dfrac{2\pi n_c}{\omega},\\ 0 & \text{for } \dfrac{2\pi n_c}{\omega} < t. \end{cases} \tag*{(7)}

Objective Function

For the sample under investigation, a set of measured wave signals uMu^M at the sensor locations xi\boldsymbol{x}_i (see Figure 1) is compared with simulated wave signals uu modeled by Equation (3). The L2L^2 misfit between the measured and simulated wave signals is a suitable choice for the objective function to detect voids:

C(γ)=12∫T∫Ω∑i=1Nr(u(x,t;γ)−uM(xi,t))2δ(x−xi) dΩ dT.(8)C(\gamma) = \frac{1}{2}\int_T \int_\Omega\sum_{i=1}^{N_r} \left(u(\boldsymbol{x},t;\gamma)-u^M(\boldsymbol{x}_i,t)\right)^2 \delta(\boldsymbol{x}-\boldsymbol{x}_i)\,\mathrm{d}\Omega\,\mathrm{d}T. \tag*{(8)}

The simulations are based on the material field γ\gamma. After evaluating the cost function, the internal material distribution γ(j)\gamma^{(j)} is updated according to Equation (2). This update relies on the derivative of the cost function with respect to γ\gamma, i.e., dC/dγ\mathrm{d}C/\mathrm{d}\gamma that is obtained using sensitivity analysis.

Sensitivity Analysis

Following the derivation of the adjoint method in [56] (for which we consider the material γ\gamma as continuous field), the sensitivity of the cost function with respect to a perturbation δγ\delta\gamma is

∇γC δγ=∫ΩKγ(u,u†) δγ dΩ,(9)\nabla_\gamma\mathcal{C}\,\delta\gamma= \int_{\Omega} K_\gamma(u,u^\dagger)\,\delta\gamma\,\mathrm{d}\Omega, \tag*{(9)}

where KγK_\gamma is a bilinear Fréchet kernel, given as

Kγ(u(x,t),u†(x,t))=−∫T(ρ0u˙†u˙+ρ0c02∇u†⋅∇u) dT.(10)K_\gamma\left(u(\boldsymbol{x},t),u^\dagger(\boldsymbol{x},t)\right)=-\int_{\mathcal{T}}\left(\rho_0\dot{u}^\dagger\dot{u}+\rho_0c_0^2\nabla u^\dagger\cdot\nabla u\right)\,\mathrm{d}T. \tag*{(10)}

Here, u†u^\dagger is the adjoint wave field underlying the adjoint wave equation

γ(x)ρ0u¨†(x,t)−∇⋅(γ(x)ρ0c02∇u†(x,t))=f†(x,t)on Ω×T(11)\gamma(\boldsymbol{x})\rho_0\ddot{u}^\dagger(\boldsymbol{x},t)-\nabla\cdot\left(\gamma(\boldsymbol{x})\rho_0c_0^2\nabla u^\dagger(\boldsymbol{x},t)\right)=f^\dagger(\boldsymbol{x},t)\quad\text{on }\Omega\times\mathcal{T} \tag*{(11)}

with the adjoint source

f†(x,t)=−∑i=1Nr(u(x,t)−uM(xi,t))δ(x−xi).(12)f^\dagger(\boldsymbol{x},t)=-\sum_{i=1}^{N_r}\left(u(\boldsymbol{x},t)-u^{M}(\boldsymbol{x}_i,t)\right)\delta(\boldsymbol{x}-\boldsymbol{x}_i). \tag*{(12)}

and temporal end conditions at tendt_{\mathrm{end}}

u†(x,tend)=u˙†(x,tend)=0on Ω.(13)u^\dagger(\boldsymbol{x},t_{\mathrm{end}})=\dot{u}^\dagger(\boldsymbol{x},t_{\mathrm{end}})=0\quad\text{on }\Omega. \tag*{(13)}

In practice, the end conditions are introduced as initial conditions by solving the adjoint equation (Equation (11)) backwards in time. As in the forward problem, homogeneous Neumann boundary conditions are enforced

n⋅∇u†=0on Γ.(14)\boldsymbol{n}\cdot\nabla u^\dagger=0\quad\text{on }\Gamma. \tag*{(14)}

Solving the forward and adjoint wave equation with FDM provides discretized wave fields. Finally, the derivative of the cost function with respect to the discretized material field at each grid point indicated by the subscript ii of the FDM is

dCdγi=∫ΩKγ(u(x,t),u†(x,t))δ(xγi−x) dΩ,(15)\frac{\mathrm{d}\mathcal{C}}{\mathrm{d}\gamma_i}=\int_{\Omega}K_\gamma\left(u(\boldsymbol{x},t),u^\dagger(\boldsymbol{x},t)\right)\delta(\boldsymbol{x}_{\gamma_i}-\boldsymbol{x})\,\mathrm{d}\Omega, \tag*{(15)}

where γi\gamma_i and xγi\boldsymbol{x}_{\gamma_i} are the material’s value and position of each grid point ii.

Constraints

As the indicator function γ\gamma is defined in the range [ϵ,1][\epsilon,1], its values are clipped to [ϵ,1][\epsilon,1] after each gradient-based update performed by the Adam optimizer (Equation (2)). Furthermore, as we are interested in detecting voids in an otherwise homogeneous material, the resulting material distribution should be binary, i.e., ϵ\epsilon or 11. This constraint is, however, not imposed as it is not strictly necessary, as empirically observed in [56, 59, 23]. Regularization techniques such as neural network parametrizations [23], total variation [60], projection [61, 62, 63], or even SIMP⁴-like penalizations [64, 65, 66], may be beneficial but are not applied in this work.

Transient Acoustic Topology Optimization

TATO in a nutshell

TATO [67, 68] aims to find the optimal design of a domain Ωd\Omega_d which is embedded in a larger domain Ω\Omega. The performance of the design is evaluated by an objective function. The optimal material distribution within Ωd\Omega_d is computed such that acoustic waves minimize the objective. The acoustic waves are excited by a point source ff. In the paper at hand, we consider the design of an acoustic black hole (see [69] for a general review and [70] for a concrete topology optimization example). The problem setup is schematically illustrated in Figure 2. Acoustic waves uu excited by the source ff travel through the domain Ω\Omega (including Ωd\Omega_d and Ωs\Omega_s). To achieve an optimal design, the waves are supposed to arrive in the domain Ωs\Omega_s with a minimum amplitude.

Image not extracted

Figure 2. Topology optimization of an acoustic black hole Ωd\Omega_d (L1d×L2dL_1^d \times L_2^d) with the aim of suppressing the sound pressure in the domain Ωs\Omega_s (L1s×L2sL_1^s \times L_2^s) at (l1s,l2s)(l_1^s,l_2^s) induced by the source ff at (l1f,l2f)(l_1^f,l_2^f). The physics of the domain Ω\Omega with dimensions L1×L2L_1 \times L_2, including Ωd\Omega_d and Ωs\Omega_s, is governed by the acoustic wave equation. The boundaries are defined as Γ=⋃i=14Γi\Gamma= \bigcup_{i=1}^{4} \Gamma_i. Modified from [71].

Physical Model

The acoustic waves are governed by the acoustic wave equation, where we rely on the modified parametrization from [45].6

κ−1(x)u¨(x,t)−∇⋅(ρ−1(x)∇u(x,t))=f(x,t)on Ω×T(16)\kappa^{-1}(x)\ddot{u}(x,t)-\nabla\cdot\left(\rho^{-1}(x)\nabla u(x,t)\right)=f(x,t)\quad\text{on }\Omega\times T \tag*{(16)}

The material is specified in terms of the inverse bulk modulus κ−1\kappa^{-1} and the inverse mass density ρ−1\rho^{-1}, which are parametrized as

ρ−1(γ(x))=1ρ1+γ(x)(1ρ2−1ρ1),(17)\rho^{-1}(\gamma(x))=\frac{1}{\rho_1}+\gamma(x)\left(\frac{1}{\rho_2}-\frac{1}{\rho_1}\right), \tag*{(17)}
κ−1(γ(x))=1κ1+γ(x)(1κ2−1κ1),(18)\kappa^{-1}(\gamma(x))=\frac{1}{\kappa_1}+\gamma(x)\left(\frac{1}{\kappa_2}-\frac{1}{\kappa_1}\right), \tag*{(18)}

with the spatially distributed indicator function γ∈[0,1]\gamma\in[0,1]. The material parameters are ρ1=1.204\rho_1=1.204 kg/m3^3 and κ1=1.419⋅105\kappa_1=1.419\cdot10^5 N/m2^2 for air, while for the solid material7 the values are ρ2=2643\rho_2=2643 kg/m3^3 and κ2=6.87⋅108\kappa_2=6.87\cdot10^8 N/m2^2. For γ=0\gamma=0 the material parameters of air are recovered, while γ=1\gamma=1 mimicks the solid material.

Again, reflecting boundary conditions in terms of homogeneous boundary conditions are employed according to Equation (4), and homogeneous initial conditions are imposed as defined in Equation (5). The specimen is excited by a sine burst, Equations (6) and (7). Note that the source term ff in Equation (16) is normalized by the inverse density ρ−1\rho^{-1}, thus carrying the unit s−2^{-2} instead of N/m2^2.

Objective Function

Based on the simulated wave pressures of a given material distribution γ\gamma, the design is assessed with the following objective function

C(γ)=1A∫T∫Ωsu(x,t;γ)2 dΩs dT.(19)C(\gamma) = \frac{1}{A} \int_{\mathcal{T}} \int_{\Omega_s} u(x,t;\gamma)^2 \, \mathrm{d}\Omega_s \, \mathrm{d}T. \tag*{(19)}

where A=∫ΩsdΩsA = \int_{\Omega_s} \mathrm{d}\Omega_s is the area of the domain of interest. Analogously to § 3.2, the current guess γ(j)\gamma^{(j)} is updated with the update rule from Equation (2) using the derivative of the objective dC/dγ\mathrm{d}C/\mathrm{d}\gamma.

Sensitivity Analysis

With the adjoint method, the sensitivity is computed equivalently to § 3.2 using Equation (9). The Fréchet kernel corresponding to the acoustic model defined in Equation (16) is

Kγ(u(x,t),u†(x,t))=−∫T((1κ2−1κ1)u˙†u˙−(1ρ2−1ρ1)∇u†⋅∇u)dT.(20)K_\gamma(u(x,t),u^\dagger(x,t)) = - \int_{\mathcal{T}} \left(\left(\frac{1}{\kappa_2} - \frac{1}{\kappa_1}\right)\dot{u}^{\dagger}\dot{u} - \left(\frac{1}{\rho_2} - \frac{1}{\rho_1}\right)\nabla u^{\dagger}\cdot\nabla u\right)\mathrm{d}T. \tag*{(20)}

The adjoint wave field u†u^\dagger obeys the adjoint wave equation

κ−1(x)u¨†(x,t)−∇⋅(ρ−1(x)∇u†(x,t))=f†(x,t)on Ω×T(21)\kappa^{-1}(x)\ddot{u}^{\dagger}(x,t) - \nabla\cdot\left(\rho^{-1}(x)\nabla u^{\dagger}(x,t)\right) = f^{\dagger}(x,t) \quad\text{on } \Omega\times\mathcal{T} \tag*{(21)}

with the adjoint source

f†(x,t)={−2u(x,t)Aif x∈Ωs,0else.(22)f^{\dagger}(x,t) = \begin{cases} -\dfrac{2u(x,t)}{A} & \text{if } x \in\Omega_s,\\ 0 & \text{else}. \end{cases} \tag*{(22)}

Again, homogeneous Neumann boundary conditions and homogeneous temporal end conditions must be employed, as in Equations (13) and (14). After solving the forward and adjoint wave fields with the FDM, the gradient with respect to γi\gamma_i considering each grid point ii can be performed following Equation (15) with Equation (20).

Constraints

To prevent mesh-dependent designs that include excessively small, and thus unmanufacturable, features, a filtering approach is utilized [72]. In particular, we apply a density filter with linear decay [73, 74]. The filter yields the following modified indicator function

γ~i=∑k∈Niw(xi−xk)γk∑k∈Niw(xi−xk).(23)\widetilde{\gamma}_i = \frac{\sum_{k\in N_i} w(x_i-x_k)\gamma_k}{\sum_{k\in N_i} w(x_i-x_k)}. \tag*{(23)}

where NiN_i is the set of grid points in the neighborhood of the grid point ii defined by the radius rfr_f, i.e., Ni={k∣∥xk−xi∥<rf}N_i = \{k \mid\lVert x_k-x_i\rVert< r_f\}. The linear decay is incorporated via the weighting function w(x)w(x), given as

w(x)={rf−∥x∥if ∥x∥≤rf,0else.(24)w(x) = \begin{cases} r_f-\lVert x\rVert& \text{if } \lVert x\rVert\le r_f,\\ 0 & \text{else}. \end{cases} \tag*{(24)}

In this work, a filter radius of 1.5 voxels (corresponding to the same filter neighborhood as in the giga-voxel optimization in [75]) is applied.

An unwanted side effect of filtering is that the designs are blurred, although sharp 0/1 designs are desired. A subsequent projection recovers an almost binary design [61, 62, 63]. Specifically, a smooth approximation of the Heaviside function H(γ~)H(\widetilde{\gamma}) achieves the projection of the filtered indicator γ~\widetilde{\gamma} to a smooth binary-like indicator γ‾~\widetilde{\overline{\gamma}}:

γ‾~=H(γ~)=tanh⁡(βη)+tanh⁡(β(γ~−η))tanh⁡(βη)+tanh⁡(β(1−η)).(25)\widetilde{\overline{\gamma}} = H(\widetilde{\gamma}) = \frac{\tanh(\beta\eta) + \tanh(\beta(\widetilde{\gamma}-\eta))}{\tanh(\beta\eta) + \tanh(\beta(1-\eta))}. \tag*{(25)}

The parameter η\eta shifts the thresholding, while β\beta adjusts the sharpness. Initially, β\beta must be kept small to allow a proper gradient flow from the projected indicator values γˉ~\tilde{\bar{\gamma}} to the indicator values γ\gamma. After initializing β=1\beta= 1, we follow a β\beta-continuation scheme [61, 62, 76], which increases β\beta by 10% every five iterations. The threshold is set as η=0.5\eta= 0.5. Note that for smaller β\beta values, the projection does not guarantee γˉ~∈[0,1]\tilde{\bar{\gamma}} \in[0, 1], which is, however, needed to ensure numerical stability (specifically the upper bound). Therefore, the filtered and projected indicator values γˉ~\tilde{\bar{\gamma}} are clipped to the interval [0,1][0, 1] at every iteration.

Modifying the indicator by filtering and projection needs to be considered during the sensitivity computation. This yields the following chain rule

dCdγ=dCdγˉ~dγˉ~dγ~∂γ~∂γ,(26)\frac{\mathrm{d}C}{\mathrm{d}\gamma} = \frac{\mathrm{d}C}{\mathrm{d}\tilde{\bar{\gamma}}} \frac{\mathrm{d}\tilde{\bar{\gamma}}}{\mathrm{d}\tilde{\gamma}} \frac{\partial\tilde{\gamma}}{\partial\gamma}, \tag*{(26)}

where the vector dC/dγˉ~\mathrm{d}C/\mathrm{d}\tilde{\bar{\gamma}} is obtained by the sensitivity analysis described in § 3.4, and the matrices dγˉ~/dγ~\mathrm{d}\tilde{\bar{\gamma}}/\mathrm{d}\tilde{\gamma} and ∂γ~/∂γ\partial\tilde{\gamma}/\partial\gamma are obtained as follows

dγˉ~dγ~=β[tanh⁡(βη)+tanh⁡(β(1−η))]cosh⁡2(β(γ~−η)),(27)\frac{\mathrm{d}\tilde{\bar{\gamma}}}{\mathrm{d}\tilde{\gamma}} = \frac{\beta}{[\tanh(\beta\eta) + \tanh(\beta(1-\eta))]\cosh^2(\beta(\tilde{\gamma}-\eta))}, \tag*{(27)}
dγ~dγk=∑i∂γ~i∂γk,(28)\frac{\mathrm{d}\tilde{\gamma}}{\mathrm{d}\gamma_k} = \sum_i \frac{\partial\tilde{\gamma}_i}{\partial\gamma_k}, \tag*{(28)}
∂γ~i∂γk={w(xi−xk)∑k∈Niw(xi−xk)if k∈Ni,0else.(29)\frac{\partial\tilde{\gamma}_i}{\partial\gamma_k} = \begin{cases} \frac{w(x_i-x_k)}{\sum_{k\in N_i}w(x_i-x_k)} & \text{if } k\in N_i,\\ 0 & \text{else.} \end{cases} \tag*{(29)}

Note that designs obtained through acoustic topology optimization are typically not robust [76]. Approaches discussed in [77, 78, 76] can amend the robustness issue, which is, however, not considered in this work.

Efficient Finite Difference Discretization on GPUs

Second-order Finite Differences

The wave equations in Equations (3) and (16) are discretized with central second-order finite differences in space and time. In one dimension, we obtain the following explicit schemes for

  • the scalar wave equation (Equation (3)):

uin+1=−uin−1+2uin+2γi(c0ΔtΔx)2((1γi+γi+1)(ui+1n−uin)−(1γi−1+γi)(uin−ui−1n))+Δt2ρ0γifin.(30)\begin{aligned} u_i^{n+1} &= -u_i^{n-1} + 2u_i^n \\ &\quad+ \frac{2}{\gamma_i}\left(\frac{c_0\Delta t}{\Delta x}\right)^2 \left( \left(\frac{1}{\gamma_i+\gamma_{i+1}}\right)(u_{i+1}^n-u_i^n) - \left(\frac{1}{\gamma_{i-1}+\gamma_i}\right)(u_i^n-u_{i-1}^n) \right) \\ &\quad+ \frac{\Delta t^2}{\rho_0\gamma_i}f_i^n. \tag*{(30)} \end{aligned}
  • and the acoustic wave equation (Equation (16)):

uin+1=−uin−1+2uin+2κi(ΔtΔx)2((1ρi+ρi+1)(ui+1n−uin)−(1ρi−1+ρi)(uin−ui−1n))+κiΔt2fin.(31)\begin{aligned} u_i^{n+1} &= -u_i^{n-1} + 2u_i^n \\ &\quad+ 2\kappa_i\left(\frac{\Delta t}{\Delta x}\right)^2 \left( \left(\frac{1}{\rho_i+\rho_{i+1}}\right)(u_{i+1}^n-u_i^n) - \left(\frac{1}{\rho_{i-1}+\rho_i}\right)(u_i^n-u_{i-1}^n) \right) \\ &\quad+ \kappa_i\Delta t^2 f_i^n. \tag*{(31)} \end{aligned}

The are straightforward to extend to two and three dimensions by expanding the spatial stencil. The spatial grid spacing is denoted as Δx\Delta x and the time step size as Δt\Delta t. Note that material points are interpolated via a harmonic mean, which is preferred for materials exhibiting discontinuities [79]. The homogeneous Neumann boundary conditions from Equation (4) are incorporated via ghost cells. The material at the ghost cells is set equal to the material at the boundary of the domain, ensuring the correct averages of γi\gamma_i in Equation (30) and ρi\rho_i in Equation (31).

Implementational Aspects

From a computational standpoint, the big advantage of the field update schemes of Equations (30) and (31) is that the computation of each grid point ii is a local operation that only involves points in its close proximity but is independent of the computation of all other locations. Such schemes benefit from the massively parallel data processing capabilities provided by modern general purpose GPU computing. To achieve maximum computational efficiency in this setting, the presented examples are computed using a CUDA® implementation. Therein, updating each grid point is handled by simultaneously executing the rule from Equations (30) and (31) in the form of CUDA® kernels. To minimize the implementational effort, only the kernels that represent a computational bottleneck are implemented in CUDA®, while the rest of the implementation uses CuPy8, a convenient Python wrapper layer on top of the standard CUDA® functionality. The corresponding implementation can be found at [51].

Alternatively, a simpler implementation can be achieved by viewing the stencils (Equations (30) and (31)) as 3×33 \times3 convolutional operations. Thereby, the stencil can be implemented as a hardcoded recurrent convolutional neural network (CNN), as proposed in [21, 23]; see Appendix § 11 for technical details. This leads to straightforward and surprisingly efficient finite difference implementations, which we will consider as a baseline for the computation times in the sequel (relying on the implementation in PyTorch [12] from [80]).

Computation Time

Considering the scalar wave equation and single precision floating-points, the computation time per time step of the finite difference forward solver versus the degrees of freedom is shown in Figure 3 for one, two, and three dimensions. All computations in this and upcoming studies were performed on an NVIDIA® A100 40 GB GPU. The CUDA® implementation (gray, black, and red lines in Figure 3) is compared to the PyTorch version based on CNNs from [23] (blue line), which is about one to two orders of magnitude slower. The two-dimensional timings were recomputed, while the three-dimensional data point is taken directly from [23]. Both represent the upper limit in terms of problem sizes, due to PyTorch’s significant memory overhead. Thus, the CUDA® implementation is clearly superior as a forward solver in both memory efficiency and speed. The graph for the CUDA® implementation consists of two parts. The computational time is independent of the number of degrees of freedom until about 2⋅1062 \cdot10^6 after which it increases with a slope of three in the double logarithmic plot. The point of increase corresponds to the maximum capacity of the L2 cache of 40 MB9 (indicated by the vertical dashed line).

Image not extracted

Figure 3. Computation time of the forward simulation versus number of degrees of freedom in one, two, and three dimensions. The timings rely on the CUDA® implementation and are compared to the timings from [23], which relied on the CNN implementation. Here, the scalar wave equation from Equation (3) is considered.

Memory-Efficient Adjoint Sensitivity Computation

To compute the sensitivity with the adjoint method in a memory-efficient manner, we consider the superposed wave field usu^s, where usu^s is the linear combination of the forward wave field uu and kk times the adjoint wave field u†u^\dagger

us(x,t)=u(x,t)+ku†(x,t).(32)u^s(x,t) = u(x,t) + k u^\dagger(x,t). \tag*{(32)}

Due to linearity of the forward and adjoint equations in uu and u†u^{\dagger}, the governing equation for the superposed solution usu^{s} can be written as

γ(x)ρ0u¨s(x,t)−∇⋅(γρ0c02∇us(x,t))=fs(x,t)(33)\gamma(\mathbf{x})\rho_{0}\ddot{u}^{s}(\mathbf{x},t)-\nabla\cdot\left(\gamma\rho_{0}c_{0}^{2}\nabla u^{s}(\mathbf{x},t)\right)=f^{s}(\mathbf{x},t) \tag*{(33)}

in FWI, while it is

κ−1u¨s(x,t)−∇⋅(ρ−1∇us(x,t))=fs(x,t)(34)\kappa^{-1}\ddot{u}^{s}(\mathbf{x},t)-\nabla\cdot\left(\rho^{-1}\nabla u^{s}(\mathbf{x},t)\right)=f^{s}(\mathbf{x},t) \tag*{(34)}

in TATO. The superposed force fsf^{s} is defined as

fs(x,t)=f(x,t)+kf†(x,t),(35)f^{s}(\mathbf{x},t)=f(\mathbf{x},t)+k f^{\dagger}(\mathbf{x},t), \tag*{(35)}

where f†f^{\dagger} is computed according to Equation (12) or Equation (22). Furthermore, superposed end conditions can be defined as

us(x,tend)=u(x,tend)+ku†(x,tend)=u(x,tend),(36)u^{s}(\mathbf{x},t_{\mathrm{end}})=u(\mathbf{x},t_{\mathrm{end}})+k u^{\dagger}(\mathbf{x},t_{\mathrm{end}})=u(\mathbf{x},t_{\mathrm{end}}), \tag*{(36)}
u˙s(x,tend)=u˙(x,tend)+ku˙†(x,tend)=u˙(x,tend),(37)\dot{u}^{s}(\mathbf{x},t_{\mathrm{end}})=\dot{u}(\mathbf{x},t_{\mathrm{end}})+k\dot{u}^{\dagger}(\mathbf{x},t_{\mathrm{end}})=\dot{u}(\mathbf{x},t_{\mathrm{end}}), \tag*{(37)}

because of the homogeneous adjoint end conditions (Equation (13)). Equivalently to the adjoint equation, the superposed equation can be solved backwards in time.

Inserting the superposed solution usu^{s} into the sensitivity kernel expression (Equation (10) or Equation (20)), yields

Kγ(us,us)=Kγ(u+ku†,u+ku†)=Kγ(u,u)⏟to be subtracted+2kKγ(u,u†)⏟gradient dC/dγi+k2Kγ(u†,u†)⏟small(38)\begin{aligned} K_{\gamma}(u^{s},u^{s})&=K_{\gamma}(u+ku^{\dagger},u+ku^{\dagger})\\ &=\underbrace{K_{\gamma}(u,u)}_{\text{to be subtracted}}+2k\underbrace{K_{\gamma}(u,u^{\dagger})}_{\text{gradient }dC/d\gamma_{i}}+k^{2}\underbrace{K_{\gamma}(u^{\dagger},u^{\dagger})}_{\text{small}} \tag*{(38)} \end{aligned}

for bilinear kernels, i.e., Kγ(u,u†)=Kγ(u†,u)K_{\gamma}(u,u^{\dagger})=K_{\gamma}(u^{\dagger},u). As u†u^{\dagger} is typically multiple orders of magnitude smaller than uu, the third term Kγ(u†,u†)K_{\gamma}(u^{\dagger},u^{\dagger}) in Equation (38) is negligible for small enough kk values. This enables an approximation of the mixed kernel K(u,u†)K(u,u^{\dagger}) needed for the sensitivity computation in Equation (9). With a suitable choice of kk, we can approximate the sensitivity Kγ(u,u†)K_{\gamma}(u,u^{\dagger}), by introducing the mixed kernel K~γ\widetilde{K}_{\gamma}, which is given as

Kγ(u,u†)≈K~γ=12k(Kγ(us,us)−Kγ(u,u)).(39)K_{\gamma}(u,u^{\dagger})\approx\widetilde{K}_{\gamma}=\frac{1}{2k}\left(K_{\gamma}(u^{s},u^{s})-K_{\gamma}(u,u)\right). \tag*{(39)}

Note that Equation (39) is only applicable in practice for time-reversible time integrators such as the second-order central differences for self-adjoint problems. Since Kγ(u,u)K_\gamma(u,u) can be integrated alongside the forward simulation, and since Kγ(us,us)K_\gamma(u^s,u^s) is independent of uu, the approximated sensitivity can be computed without saving the forward wave field uu. The pseudo-code illustrated in Algorithm 1 computes the cost and gradient for a material distribution.

Algorithm 1 Memory-efficient mixed kernel computation K~γ(u,u†)≈Kγ(u,u†)\widetilde{K}_\gamma(u,u^\dagger) \approx K_\gamma(u,u^\dagger)

Require: measurement uMu^\mathcal{M}, indicator γ\gamma, source fnf^n, factor kk, number of time steps NN

  1. initialize cost C=0C=0 kernel K~γ=0\widetilde{K}_\gamma=0

  2. initialize u0=0u^0=0, u1=0u^1=0

solve forward problem, compute adjoint force & increment kernel (forward in time)

  1. for time step n=1,…,N−1n=1,\ldots,N-1 do

  2. un+1←propagateStep⁡(un−1,un,fn,γ)u^{n+1}\leftarrow\operatorname{propagateStep}(u^{n-1},u^n,f^n,\gamma) ▷ (cf. Equations (30) and (31) including BCs)

  3. f†n+1←computeAdjointSource⁡(un+1,uM)f^{\dagger n+1}\leftarrow\operatorname{computeAdjointSource}(u^{n+1},u^\mathcal{M}) ▷ (cf. Equations (12) and (22))

  4. C←C+incrementCost⁡(un+1,uM)C\leftarrow C+\operatorname{incrementCost}(u^{n+1},u^\mathcal{M}) ▷ (cf. Equations (8) and (19))

  5. K~γ←K~γ−incrementKernel⁡(un+1)\widetilde{K}_\gamma\leftarrow\widetilde{K}_\gamma-\operatorname{incrementKernel}(u^{n+1}) ▷ cf. Equations (10), (20) and (39) (−Kγ(u,u)-K_\gamma(u,u))

  6. delete un−1u^{n-1}

  7. end for

  8. scale adjoint force f†n+1←kf†n+1f^{\dagger n+1}\leftarrow kf^{\dagger n+1} ▷ cf. Equation (35)

  9. initialize usN=uNu^{sN}=u^N, usN−1=uN−1u^{sN-1}=u^{N-1}

solve superposed problem & increment kernel (backward in time)

  1. for time step n=N−1,…,1n=N-1,\ldots,1 do

  2. usn−1←propagateStep⁡(usn−1,usn,f†n+fn,γ)u^{s n-1}\leftarrow\operatorname{propagateStep}(u^{s n-1},u^{s n},f^{\dagger n}+f^n,\gamma) ▷ (cf. Equations (30) and (31) including BCs)

  3. K~γ←K~γ+incrementKernel⁡(usn−1)\widetilde{K}_\gamma\leftarrow\widetilde{K}_\gamma+\operatorname{incrementKernel}(u^{s n-1}) ▷ cf. Equations (10), (20) and (39) (Kγ(us,us)K_\gamma(u^s,u^s))

  4. delete usn+1u^{s n+1}

  5. end for

  6. return CC, K~γ\widetilde{K}_\gamma

Given the current indicator γ\gamma, the forward problem is solved (line 4), yielding uu and f†f^\dagger (line 5). Simultaneously, the cost is incremented (line 6), and the contribution of Kγ(u,u)K_\gamma(u,u) is computed and added to K~γ\widetilde{K}_\gamma (line 7). After scaling the adjoint forces, according to Equation (35), the superposed problem (Equation (33) or Equation (34)) is solved for usu^s (line 13). During the time integration, the contribution of Kγ(us,us)K_\gamma(u^s,u^s) is computed and added to K~γ\widetilde{K}_\gamma (line 14). After all time steps, the approximated mixed kernel K~γ\widetilde{K}_\gamma from Equation (39) is retrieved, which approximates the sensitivity from Equation (9). Importantly, note, that after each incremented time step correspondingly un−1u^{n-1} (line 8) and usn+1u^{s n+1} (line 15) are deleted, meaning that at most four solution vectors are needed (un−1/usn−1u^{n-1}/u^{s n-1}, un/usnu^n/u^{s n}, un+1/usn+1u^{n+1}/u^{s n+1}, K~γ\widetilde{K}_\gamma), which all have the size of the number of degrees of freedom.

Approximation Quality

The quality of the approximation depends on the choice of the parameter kk. However, it is easy to show that the range of permissible values of kk is relatively large for FWI and TATO. To demonstrate this, we consider the FWI example illustrated in Figure 4a, and the TATO problem illustrated in Figure 4c. For details, see Table 110. Assuming homogeneous material distributions of γ(x)=1\gamma(x)=1 for FWI and γ(x)=0\gamma(x)=0 for TATO, reference sensitivities were computed with double floating-point precision using the standard adjoint sensitivity method, described in § 3.2 and § 3.4. The sensitivity fields are depicted in Figure 4b and Figure 4d.

L1L_1L2L_2n1n_1n2n_2NNΔt\Delta tffncn_cψ0\psi_0
FWI (Figure 4a)0.02 m0.02 m2512513 2007.5⋅10−97.5 \cdot10^{-9} s10610^6 Hz2101210^{12} N/m2^2
TATO (Figure 4c)9 m9 m3633631 8003.4⋅10−23.4 \cdot10^{-2} s650 Hz210210^2 s−2^{-2}

Table 1. Problem parameters of the two-dimensional FWI and TATO defined in Figure 4, including the domain size L1×L2L_1 \times L_2, grid points n1×n2n_1 \times n_2, number of time steps NN, time step size Δt\Delta t, and source parameters ff, ncn_c, ψ0\psi_0 defined according to Equations (6) and (7). For the detailed sensor and source arrangement, see [51].

Image not extracted

Figure 4. Reference gradients for FWI (left) and TATO (right). During the gradient computation homogeneous material distributions of correspondingly γ(x)=1\gamma(x) = 1 and γ(x)=0\gamma(x) = 0.

Performing the gradient computation with the memory-efficient adjoint sensitivity approximation, summarized by Algorithm 1, for multiple kk values and using the reference gradients from Figure 4, the relative error with respect to kk is illustrated in Figure 5. The relative absolute mean squared error is shown for both single floating-point precision (black) and double floating-point precision (blue). The dashed line indicates a 5% threshold, leading to a range of about seven orders of magnitude, resulting in a relative error of less than 5%. Double precision provides an even greater range, as the errors for smaller kk values are slower to rise, due to smaller overflow/underflow problems. The increasing error for larger kk values is caused by neglecting the term k2Kγ(u†,u†)k^2 K_\gamma(u^\dagger,u^\dagger) in Equation (39).

Image not extracted

(a) FWI

Image not extracted

(b) TATO

Figure 5. Relative error magnitude versus the parameter kk for floating and double precision. Both graphs span 22 orders of magnitude of kk.

The approximate sensitivities for five different kk values — (1012,1013,1016,1018,1019)(10^{12}, 10^{13}, 10^{16}, 10^{18}, 10^{19}) for FWI and (10−4,10−3,10−1,101,102)(10^{-4}, 10^{-3}, 10^{-1}, 10^1, 10^2) for TATO — are illustrated in Figure 6 and Figure 7. The qualitative comparison to the reference gradients from Figure 4b and Figure 4d highlights the effect of the overflow/underflow error for small kk and the approximation error for large kk values11. Yet, still, the robustness of the approximation over a large range of kk values is emphasized. Finding an appropriate kk value can be achieved during the first sensitivity computation of the optimization, in which large kk values are chosen and decreased until overflow/underflow errors become apparent. An overflow/underflow error can unambiguously be identified by comparing the single and double precision computations, which would lead to different results (unlike when the approximation errors for large kk values arise). The smallest possible kk value before overflow/underflow is an appropriate choice. Note that this calibration does not require the true gradient as ground truth.

Image not extracted

(a) k=1012k = 10^{12}

Image not extracted

(b) k=1013k = 10^{13}

Image not extracted

(c) k=1016k = 10^{16}

Image not extracted

(d) k=1018k = 10^{18}

Image not extracted

(e) k=1019k = 10^{19}

Figure 6. FWI: approximate sensitivity with different kk values (reference sensitivity given in Figure 4b)

Image not extracted

(a) K~(u,u†)\widetilde{K}(u,u^\dagger) with k=10−4k = 10^{-4}

Image not extracted

(b) K~(u,u†)\widetilde{K}(u,u^\dagger) with k=10−3k = 10^{-3}

Image not extracted

(c) K~(u,u†)\widetilde{K}(u,u^\dagger) with k=10−1k = 10^{-1}

Image not extracted

(d) K(u,u†)K(u,u^\dagger) with k=101k = 10^{1}

Image not extracted

(e) K(u,u†)K(u,u^\dagger) with k=102k = 10^{2}

Figure 7. TATO: approximate sensitivity with different kk values (reference sensitivity given in Figure 4d)

Furthermore, we have empirically observed that the kk value selected for the first optimization iteration is equally applicable to later stages of the optimization procedure. To illustrate this, we

consider an optimized structure (Figure 8b) for the TATO problem defined by Figure 4c. Computing the gradient at the optimum and comparing it with a reference gradient using the standard adjoint sensitivity method yields a slightly worse approximation, as seen by Figure 8a (gray line). Nevertheless, the range of appropriate kk is only slightly reduced and not shifted drastically, thus retaining the validity of a kk tuned at the first iteration.

Image not extracted

Figure 8. (a) Effects on approximation quality (b) Optimized structure given by Figure 4c with the corresponding wave field at t=4.42⋅10−2 st = 4.42 \cdot10^{-2}\,\mathrm{s}

Figure 8: Number of time steps and indicator function can influence quality of the approximation negatively. Results were all computed with single floating-point precision.

Lastly, the number of time steps also influences the quality of the approximation. This is illustrated for N=1 200N = 1\,200, N=4 800N = 4\,800 and N=9 600N = 9\,600 time steps in Figure 8a. The effect is, however, limited. More importantly, for the problem at hand, N=9 600N = 9\,600 time steps was the maximum of time steps possible with the standard adjoint sensitivity method, as the memory requirement scales with the number of time steps. By contrast, the proposed memory-efficient method’s memory usage is independent of the number of time steps. Thus, even if the quality would deteriorate for very large numbers of time steps (N≫10 000N \gg10\,000), the standard adjoint sensitivity computation would not even be applicable.

Computation Time

We now consider the computation time, as previously for the forward simulation in Figure 3. A single sensitivity computation is composed of one forward and one adjoint solve. Figure 9 illustrates how the standard gradient computation (gray) described in § 3.2 is limited in terms of memory requirements and thereby barely reaches 10610^6 degrees of freedom12. This is alleviated by the memory-efficient adjoint sensitivity computation, that is summarized by Algorithm 1. Here, the gradient for up to 10910^9 degrees of freedom can be computed. The theoretical limit without implementational overheads is the memory footprint of four times the number of degrees of freedom, which with the 40 GB of memory yields 40 GB /4/4 B = 2.5⋅1092.5 \cdot10^9 parameters for single floating-point precision. It is four times, due to the three solution grids for the finite difference scheme and one grid for the sensitivity analysis, as laid out in § 5.

Image not extracted

Figure 9. Computation time of the gradient computation (including one forward and one adjoint solve) versus number of degrees of freedom in one, two, and three dimensions. The timings rely on the CUDA® implementation and are compared to the timings from [23], which relied on the CNN implementation. Here, the scalar wave equation from Equation (3) and the sensitivity analysis for FWI from § 3.2 are considered.

Also, in terms of computation time, the memory-efficient scheme outperforms the standard computation scheme, which is most likely connected to more efficient memory access patterns linked to the lower memory usage. Furthermore, take note that the gradient computations are about one order of magnitude more expensive than the corresponding forward simulations in Figure 3, despite consisting only of one forward computation and one adjoint computation. This is also linked to less efficient memory access patterns, which might be improved through more sophisticated implementations.

In Figure 9, the memory-efficient scheme (black) is compared to the CNN implementation (blue). For low numbers of degrees of freedom, the CNN version is slightly better13 than the memory-efficient scheme, which is probably linked to sophisticated implementations by PyTorch. For larger problems, however, at ∼106\sim10^{6}, the CNN scheme is one order of magnitude slower. This is also the limit in terms of the size of what is possible in PyTorch, due to the extensive memory overhead of PyTorch. Thus, also in the gradient computation, the CUDA® version (with the memory-efficient adjoint scheme) is superior. Nevertheless, the PyTorch CNN idea is beneficial for fast prototyping when working with smaller problems.

Optimization Results

To demonstrate the memory-efficient adjoint sensitivity in optimization scenarios, we consider an FWI example in three dimensions (§ 6.1) and a TATO example in two dimensions (§ 6.2).

Full Waveform Inversion

We investigate the three-dimensional structure of a cylindrical probe [83]. The task is to recover the internal voids seen in Figure 10. The geometry was obtained using computed tomography (CT). In the following, we aim to reconstruct the CT scan using FWI (which, in practice, would be a much cheaper alternative to generate a volumetric image of the sample). The full structure is characterized by the parameters in Table 2. As the inversion with FWI is a numerically very challenging task, subparts of the structure are considered during the inversion. The upper eighth, quarter, half, and full structures are considered. The smaller substructures with n3=124n_3 = 124, n3=246n_3 = 246 and n3=490n_3 = 490 are simulated with correspondingly fewer time steps: N=1300N = 1300, N=2400N = 2400, and N=3800N = 3800. To improve the direct comparison between the inversion results of the four structures, an additional void is added at the center, namely a hole in cubical shape composed of 16×16×1616 \times16 \times16 voxels.

problem parameters
L1L_1L2L_2L3L_3ffncn_cψ0\psi_0
0.05 m0.05 m0.0973 m2⋅1062 \cdot10^6 Hz2101210^{12} N/m2^2
discretization & optimization parameters
n1n_1n2n_2n3n_3NNΔt\Delta tα\alpha
5035039767 2009⋅10−99 \cdot10^{-9} s0.2

Table 2. Problem parameters of the three-dimensional FWI problem based on the CT-scanned structure from Figure 10 and measurement setup defined in Figure 11a and Figure 11b. This includes the domain size L1×L2×L3L_1 \times L_2 \times L_3, the grid points n1×n2×n3n_1 \times n_2 \times n_3, number of time steps NN, time step size Δt\Delta t, source parameters ff, ncn_c, ψ0\psi_0, and learning rate α\alpha for Adam.

Image not extracted

Figure 10. Cylindrical probe with internal voids, obtained with a CT-scan from [83]. The structure is divided into an eighth, a quarter, and a half substructure, as the circular slices indicate.

For the inversion, 33×3333 \times33 sensors are placed at the top and bottom surfaces of the cylinder, according to Figure 11. In total, four sources are applied at the bottom surface. The cylinder is simulated with the finite difference discretization described in § 4. The cylindrical shape is handled in an embedded sense [84, 85], i.e., it is embedded in a larger rectangular cuboid, as illustrated in Figure 11 for the top and bottom surfaces. The fictitious domain, indicated in gray, is handled by setting γ=ϵ\gamma= \epsilon. Only the internal geometry shown in black is modified during the inversion procedure.

Image not extracted

Figure 11. (a) Sensor placement on top surface. (b) Source and sensor placement on bottom surface. Measurement setup. The two sensor 33×3333 \times33 grids on top and bottom surfaces are indicated in blue, while the 2×22 \times2 sources (in red) are placed at the bottom.

The resulting FWI results are provided in Figure 12 for the one eighth (1/81/8), one quarter (1/41/4), and one half (1/21/2) substructures. The computational time per iteration for each of the structures is provided in Table 4 together with the associated number of degrees of freedom. Note that all computations are impossible to carry out without employing Algorithm 1 as even the smallest domain size necessitates 31 373 116 degrees of freedom. However, as seen in Figure 12c, the inversion quality deteriorates for very large problem sizes. This is even clearer when viewing the central slices of each of the substructures (now including the full structure (1/11/1)) in Figure 13, where the centered cuboid aids in a direct qualitative comparison. As can be seen in Figure 13d, we were unable to perform a successful inversion for the full structure, since the wave field information from waves traveling through the entire cylinder is insufficient. Yet, these are issues related to the well-posedness of FWI and not directly a result of the approximation used in Algorithm 1. Instead, the main advantage of Algorithm 1 is that it enables to investigate FWI for such large structures on fast GPU implementations at all.

Image not extracted

Figure 12. Internal voids recovered with FWI (thresholded at γ=0.1\gamma= 0.1) in comparison to the ground truth

Figure 13. Central slices comparing results obtained from inversion and the true internal voids for different substructures defined according to Figure 10. The inversion of (1/1) only ran for ten iterations without successful convergence.

Fig. 6Fig. 7 & 8bFig. 13aFig. 13bFig. 13cFig. 13dFig. 14
dofs63 001131 76931 373 11662 240 214123 974 410246 936 78414 604 309
NN3 2001 8001 3002 4003 8007 20017 184
sources4144441
iter. time [s]∼1.8⋅10−4\sim1.8 \cdot10^{-4}∼2.0⋅10−4\sim2.0 \cdot10^{-4}∼1.5⋅10−2\sim1.5 \cdot10^{-2}∼3.3⋅10−2\sim3.3 \cdot10^{-2}∼6.6⋅10−2\sim6.6 \cdot10^{-2}∼1.6⋅10−1\sim1.6 \cdot10^{-1}∼1.5⋅10−3\sim1.5 \cdot10^{-3}
iters.50300200200100(10)300
total time [s]∼115 s\sim115\,\mathrm{s}∼109 s\sim109\,\mathrm{s}∼4.3 h\sim4.3\,\mathrm{h}∼17.6 h\sim17.6\,\mathrm{h}∼27.9 h\sim27.9\,\mathrm{h}(∼12.8 h\sim12.8\,\mathrm{h})∼2.1 h\sim2.1\,\mathrm{h}

Table 4. Computation times for all considered cases obtained on an NVIDIA® A100 40 GB GPU. The time is given per iteration (per time step) and, in the case of FWI per source, i.e., divided by four in all considered cases. The total time is obtained by multiplying the number of time steps NN, number of sources, iteration time, and number of iterations.

Transient Acoustic Topology Optimization

We consider optimizing an acoustic black hole, following Figure 2. The corresponding problem parameters are given in Table 3. In addition to performing an optimization that suppresses the acoustic pressure in the blue domain Ωs\Omega_{\mathrm{s}} in Figure 2, an optimization that amplifies the pressure is also performed. To this end, the signs of Equations (19) and (22) are simply inverted. While Figure 14a shows the design achieving noise suppression, the design of Figure 14b leads to amplification.

problem parameters
L1L_1L2L_2ffncn_{\mathrm{c}}ψ0\psi_0
18 m9 m5000 Hz2102 s−210^2\ \mathrm{s}^{-2}
discretization & optimization parameters
n1n_1n2n_2NNΔt\Delta tα\alpha
5403270317 184∼4.58⋅10−6 s\sim4.58 \cdot10^{-6}\ \mathrm{s}0.1

Table 3. Problem parameters of the two-dimensional TATO problem defined in Figure 2. This includes the domain size L1×L2L_1 \times L_2, the grid points n1×n2n_1 \times n_2, number of time steps NN, time step size Δt\Delta t, source parameters ff, ncn_{\mathrm{c}}, ψ0\psi_0, and learning rate α\alpha for Adam.

Image not extracted

Figure 14. Optimized designs (with filter radius 1.5) suppressing (left) or amplifying (right) the acoustic pressures inside the blue square (Ωs\Omega_s).

However, the obtained designs are clearly unmanufacturable. Larger filter radii (see Appendix § 12 for examples) amend the manufacturability issue to a certain extent, but also yield less desirable designs in terms of the objective function (Equation (19)). More sophisticated options are incorporating robustness into the optimization [77, 78, 76] or extending it with structural integrity through a simultaneous compliance minimization [86, 66]. Ensuring manufacturability in TATO is, however, out of the scope of the present work.

To illustrate how the optimized structures interact with the wave fields and achieve their corresponding goals, snapshots before and after the first wave fronts arrive at Ωs\Omega_s are considered. Figure 15 visualizes these snapshots (at t≈3.66⋅10−2 st \approx3.66 \cdot10^{-2}\,\mathrm{s}, t≈5.00⋅10−2 st \approx5.00 \cdot10^{-2}\,\mathrm{s}, t≈6.41⋅10−2 st \approx6.41 \cdot10^{-2}\,\mathrm{s}, and t≈7.78⋅10−2 st \approx7.78 \cdot10^{-2}\,\mathrm{s}) for the suppressing structure (from Figure 14a), the amplifying structure (from Figure 14a), and no structure, i.e., the situation with only air. At t>5⋅10−2 st > 5 \cdot10^{-2}\,\mathrm{s}, noise is clearly suppressed in Figure 15d and amplified in Figure 15l in comparison to Figure 15h. This becomes even clearer by considering the transient response of the integrated acoustic pressures within Ωs\Omega_s, as shown by Figure 16.

Wave fields interacting with designs at four times for suppression, no structure, and amplification

Figure 15. Wave fields interacting with designs: suppression (top), no structure (center), amplification (bottom)

Figure 16. Transient response of integrated acoustic pressures in the domain of interest (blue domain in Figure 2 and Figure 14). The pressures are normalized by the maximum pressure u~\tilde{u} reached by the transient response without a structure (Figure 15e to Figure 15h).

The corresponding computation time is again provided in Table 4 together with the degrees of freedom.

Conclusion

In this paper, we propose a memory-efficient approach to compute the adjoint gradient for dynamic optimization problems, such as full waveform inversion (FWI) and transient acoustic topology optimization (TATO). The approach exploits the linearity of the underlying problem by using a superposition of the adjoint wave field with the forward computation. Based on this approach, we avoid the need to save the forward wave field for the gradient computation. The memory requirement is reduced from the number of degrees of freedom times the number of time steps to only the number of degrees of freedom. It thereby breaks the memory bound of classical adjoint approaches for transient problems which has been a pressing issue for efficient GPU-based implementations.

The sensitivity approximation comes with no additional computational effort in comparison to the standard adjoint sensitivity method. Since the approach relies on the time-reversibility of the forward simulation, it is limited to self-adjoint problems and time-reversible integration schemes. The approximation quality depends on a hyperparameter kk, but the quality of the inversion is only mildly dependent on its choice as the range of the admissible values of kk spans multiple orders of magnitude. Moreover, kk can easily be estimated prior to the optimization, by reducing kk in the first optimization as much as possible until the computed sensitivity starts changing. A verification can also be performed by comparing a single precision computation with a double precision computation.

Relying on an efficient GPU-based finite difference solver (available at [51]), we have successfully applied the approximation to the inverse problem of FWI (up to 123 974 410 degrees of freedom in three dimensions) and the optimization problem of TATO (up to 14 604 309 degrees of freedom in two dimensions). Sensitivities were computed for problems up to ∼109\sim10^{9} parameters. The theoretical limit, however, lies at the memory capacity divided by the memory footprint of four times the number of degrees of freedom, i.e., 40 GB /4/4 B = 2.5⋅1092.5 \cdot10^{9} parameters for an NVIDIA® A100 GPU with 40 GB. This can even be pushed further with multi-GPU implementations14 or potentially larger memory capacities of future GPU architectures15. Beyond the direct applicability in standard dynamic optimization, the memory-efficient sensitivity approximation is useful in the so-called neural reparametrizations — also known as neural topology optimization (see [90, 91] for inverse problems in general, [92, 23, 93, 94, 95, 96] for FWI, [97, 98, 99, 100, 101, 102, 103, 104, 105] for topology optimization16 in general, and [71] for acoustic topology optimization), where neural networks are employed to reparametrize design variables. Neural networks are preferably implemented on GPUs, where the memory capacity is a limitation. Thus, GPU-based forward solvers and memory-efficient sensitivity computations are of practical relevance to these techniques. Finally, we expect that our approach can be combined with random boundaries proposed in [37, 38] to mimic absorbing boundary conditions.

Acknowledgements

The authors gratefully acknowledge the funding provided by the Deutsche Forschungsgemeinschaft under Project no. 438252876, Grant KO 4570/1-2 which supports Leon Herrmann and RA 624/29-2, which supports Tim Bürchner. Furthermore, we thank the Georg Nemetschek Institut (GNI) for the support provided in the joint research project DeepMonitor.

Declarations

Conflict of interest No potential conflict of interest was reported by the authors.

Replication of Results

We provide a PyTorch and CUDA® implementation of the finite difference solver and the memory-efficient adjoint sensitivity method in [51]. Together with the CNN implementation from [80], all results can be replicated.

Finite Difference Stencils as Convolutional Kernels

The finite difference stencils from Equations (30) and (31) can be rewritten in terms of convolutions. Consider the wave equation discretization from Equation (30), rewritten as

un+1=−un−1+2un+2γ(11γ∗Kγ0(un∗Ku0)−11γ∗Kγ1(un∗Ku1))+Δt22ρ0fn,(40)\begin{aligned} u^{n+1} &= -u^{n-1} + 2u^n \\ &\quad+ \frac{2}{\gamma}\left(\frac{1}{\frac{1}{\gamma} * K_{\gamma_0}}(u^n * K_{u_0}) - \frac{1}{\frac{1}{\gamma} * K_{\gamma_1}}(u^n * K_{u_1})\right) \\ &\quad+ \frac{\Delta t^2}{2\rho_0}f^n, \tag*{(40)} \end{aligned}

where the corresponding kernels are

Kγ0=(011),K_{\gamma_0} = \begin{pmatrix} 0 & 1 & 1 \end{pmatrix},
Ku0=(0−(c0ΔtΔx)2(c0ΔtΔx)2),K_{u_0} = \begin{pmatrix} 0 & -\left(\frac{c_0\Delta t}{\Delta x}\right)^2 & \left(\frac{c_0\Delta t}{\Delta x}\right)^2 \end{pmatrix},
Kγ1=(110),K_{\gamma_1} = \begin{pmatrix} 1 & 1 & 0 \end{pmatrix},
Ku1=(−(c0ΔtΔx)2(c0ΔtΔx)20).K_{u_1} = \begin{pmatrix} -\left(\frac{c_0\Delta t}{\Delta x}\right)^2 & \left(\frac{c_0\Delta t}{\Delta x}\right)^2 & 0 \end{pmatrix}.

increase drastically, as seen by the current NVIDIA® H200 (141 GB) and NVIDIA® Blackwell Ultra (288 GB), but also the announced NVIDIA® Rubin Ultra (1 TB). This would correspondingly expand the possible degrees of freedom from 1.5⋅1091.5 \cdot10^9 to ∼8.8⋅109\sim8.8 \cdot10^9 (141 GB), 1.8⋅10101.8 \cdot10^{10} and ∼6.3⋅1010\sim6.3 \cdot10^{10} (1 TB).

These kernels act on the solution, force, and indicator vectors

un=(ui−1n,uin,ui+1n),fn=(fi−1n,fin,fi+1n),γ=(γi−1,γi,γi+1).(41)\mathbf{u}^{n} = (u_{i-1}^{n},u_{i}^{n},u_{i+1}^{n}), \\ \mathbf{f}^{n} = (f_{i-1}^{n},f_{i}^{n},f_{i+1}^{n}), \\ \gamma= (\gamma_{i-1},\gamma_{i},\gamma_{i+1}). \tag*{(41)}

Through the formulation through convolutions in Equation (40), the forward solver can be implemented as a recurrent CNN, as outlined in detail in [23] with a corresponding code available at [80].

Designs with Filter Radius 12

Performing the same optimization used for the designs in Figure 14, but with a filter radius of 12 instead of 1.5, yields the structures in Figure 1717. Corresponding transient responses — less pronounced than the ones in Figure 16 — are given by Figure 18.

Optimized designs with filter radius 12

Figure 17. Optimized designs (with filter radius 12) suppressing (left) or amplifying (right) the acoustic pressures inside the blue square (Ωs\Omega_s)

Image not extracted

Figure 18. Transient response of integrated acoustic pressures in the domain of interest (blue domain in Figure 2 and Figure 17). The pressures are normalized by the maximum pressure u~\tilde{u} reached by the transient response without a structure (Figure 15e to Figure 15h). In contrast to Figure 18, the designs are filtered with a radius of 12.

References

References

  1. [1]David Michéa and Dimitri Komatitsch. Accelerating a three-dimensional finite-difference wave propagation code using GPU graphics cards: Accelerating a wave propagation code using GPUs. Geophysical Journal International, pages no–no, May 2010. ISSN 0956540X, 1365246X. doi: 10.1111/j.1365-246X.2010.04616.x. URL https://academic.oup.com/gji/article-lookup/doi/10.1111/j.1365-246X.2010.04616.x.
  2. [2]Brian Hamilton and Craig J. Webb. Room acoustics modelling using GPU-accelerated finite difference and finite volume methods on a face-centered cubic grid. June 2013. URL https://hgpu.org/?p=9655.
  3. [3]Chuang-Chao Ye, Peng-Jun-Yi Zhang, Zhen-Hua Wan, Rui Yan, and De-Jun Sun. Accelerating CFD simulation with high order finite difference method on curvilinear coordinates for modern GPU clusters. Advances in Aerodynamics, 4(1):7, February 2022. ISSN 2524-6992. doi: 10.1186/s42774-021-00098-3. URL https://doi.org/10.1186/s42774-021-00098-3.
  4. [4]Martin Schweiger. GPU-Accelerated Finite Element Method for Modelling Light Transport in Diffuse Optical Tomography. International Journal of Biomedical Imaging, 2011:403892, 2011. ISSN 1687-4196. doi: 10.1155/2011/403892.DOI
  5. [5]Erik A. Träff, Anton Rydahl, Sven Karlsson, Ole Sigmund, and Niels Aage. Simple and efficient GPU accelerated topology optimisation: Codes and applications. Computer Methods in Applied Mechanics and Engineering, 410:116043, May 2023. ISSN 0045-7825. doi: 10.1016/j.cma.2023.116043. URL https://www.sciencedirect.com/science/article/pii/S0045782523001676.DOI
  6. [6]Michael Afanasiev, Christian Boehm, Martin van Driel, Lion Krischer, Max Rietmann, Dave A May, Matthew G Knepley, and Andreas Fichtner. Modular and flexible spectral-element waveform modelling in two and three dimensions. Geophysical Journal International, 216(3):1675–1692, March 2019. ISSN 0950-540X, 1365-246X. doi: 10.1093/gji/ggy469. URL https://academic.oup.com/gji/article/216/3/1675/5174970.DOI
  7. [7]Jinghen Xu, Haohuan Fu, Lin Gan, Chao Yang, Wei Xue, Shizhen Xu, Wenlai Zhao, Xinliang Wang, Bingwei Chen, and Guangwen Yang. Generalized GPU Acceleration for Applications Employing Finite-Volume Methods. In 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pages 126–135, May 2016. doi: 10.1109/CCGrid.2016.30. URL https://ieeexplore.ieee.org/document/7515679.DOI
  8. [8]Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. ISBN 0-262-03561-8. URL http://www.deeplearningbook.org.
  9. [9]Christopher M. Bishop and Hugh Bishop. Deep Learning: Foundations and Concepts. Springer International Publishing, Cham, 2024. ISBN 978-3-031-45467-7 978-3-031-45468-4. doi: 10.1007/978-3-031-45468-4. URL https://link.springer.com/10.1007/978-3-031-45468-4.DOI
  10. [10]Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viegas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems, 2016. URL https://arxiv.org/abs/1603.04467.
  11. [11]James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  12. [12]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köp̄f, Edward Yang, Zach DeVito, Martin Raison, Al̄ykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library, December 2019. URL http://arxiv.org/abs/1912.01703.
  13. [13]Stefan Kollmannsberger, Davide D’Angella, Moritz Jokeit, and Leon Herrmann. Deep Learning in Computational Mechanics: An Introductory Course, volume 977 of Studies in Computational Intelligence. Springer International Publishing, Cham, 2021. ISBN 978-3-030-76586-6 978-3-030-76587-3. doi: 10.1007/978-3-030-76587-3. URL https://link.springer.com/10.1007/978-3-030-76587-3.DOI
  14. [14]Leon Herrmann and Stefan Kollmannsberger. Deep learning in computational mechanics: a review. Computational Mechanics, January 2024. ISSN 0178-7675, 1432-0924. doi: 10.1007/s00466-023-02434-4. URL https://link.springer.com/10.1007/s00466-023-02434-4.DOI
  15. [15]Jun Takeuchi and Yukio Kosugi. Neural network representation of finite element method. Neural Networks, 7 (2):389–395, January 1994. ISSN 0893-6080. doi: 10.1016/0893-6080(94)90031-0. URL https://www.sciencedirect.com/science/article/pii/0893608094900310.DOI
  16. [16]P. Ramuhalli, L. Udpa, and S.S. Udpa. Finite-element neural networks for solving differential equations. IEEE Transactions on Neural Networks, 16(6):1381–1392, November 2005. ISSN 1941-0093. doi: 10.1109/TNN.2005.857945. URL https://ieeexplore.ieee.org/document/1528518.DOI
  17. [17]Houpu Yao, Yi Ren, and Yongming Liu. FEA-Net: A Deep Convolutional Neural Network With PhysicsPrior For Efficient Data Driven PDE Learning. In AIAA SciTech 2019 Forum, San Diego, California, January 2019. American Institute of Aeronautics and Astronautics. ISBN 978-1-62410-578-4. doi: 10.2514/6.2019-0680. URL https://arc.aiaa.org/doi/10.2514/6.2019-0680.DOI
  18. [18]Houpu Yao, Yi Gao, and Yongming Liu. FEA-Net: A physics-guided data-driven model for efficient mechanical response prediction. Computer Methods in Applied Mechanics and Engineering, 363:112892, May 2020. ISSN 0045-7825. doi: 10.1016/j.cma.2020.112892. URL https://www.sciencedirect.com/science/article/pii/S0045782520300748.DOI
  19. [19]Chanwook Park, Ye Lu, Sourav Saha, Tianju Xue, Jiachen Guo, Satyajit Mojumder, Daniel W. Apley, Gregory J. Wagner, and Wing Kam Liu. Convolution hierarchical deep-learning neural network (C-HiDeNN) with graphics processing unit (GPU) acceleration. Computational Mechanics, 72(2):383–409, August 2023. ISSN 1432-0924. doi: 10.1007/s00466-023-02329-4. URL https://doi.org/10.1007/s00466-023-02329-4.
  20. [20]R.K. Mishra and P.S. Hall. NEFDTD concept. IEEE Transactions on Neural Networks, 16(2):484–490, March 2005. ISSN 1941-0093. doi: 10.1109/TNN.2004.841799. URL https://ieeexplore.ieee.org/document/1402508.
  21. [21]Alan Richardson. Seismic Full-Waveform Inversion Using Deep Learning Tools and Techniques, January 2018. URL http://arxiv.org/abs/1801.07232. arXiv:1801.07232.
  22. [22]Jian Sun, Zhan Niu, Kristoffer A. Innanen, Junxiao Li, and Daniel O. Trad. A theory-guided deep-learning formulation and optimization of seismic waveform inversion. GEOPHYSICS, 85(2):R87–R99, March 2020. ISSN 0016-8033, 1942-2156. doi: 10.1190/geo2019-0138.1. URL https://library.seg.org/doi/10.1190/geo2019-0138.1.DOI
  23. [23]Leon Herrmann, Tim Bü̈rchner, Felix Dietrich, and Stefan Kollmannsberger. On the use of neural networks for full waveform inversion. Computer Methods in Applied Mechanics and Engineering, 415:116278, October 2023. ISSN 00457825. doi: 10.1016/j.cma.2023.116278. URL https://linkinghub.elsevier.com/retrieve/pii/S0045782523004024.DOI
  24. [24]R.-E. Plessix. A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophysical Journal International, 167(2):495–503, November 2006. ISSN 0956540X, 1365246X. doi: 10.1111/j.1365-246X.2006.02978.x. URL https://academic.oup.com/gji/article-lookup/doi/10.1111/j.1365-246X.2006.02978.x.DOI
  25. [25]A. Fichtner, H.-P. Bunge, and H. Igel. The adjoint method in seismology: I. theory. Physics of the Earth and Planetary Interiors, 157(1):86–104, 2006. ISSN 0031-9201. doi: https://doi.org/10.1016/j.pepi.2006.03.016. URL https://www.sciencedirect.com/science/article/pii/S0031920106001051.
  26. [26]A. Fichtner, H.-P. Bunge, and H. Igel. The adjoint method in seismology—II. applications: traveltimes and sensitivity functionals. Physics of the Earth and Planetary Interiors, 157(1):105–123, 2006. ISSN 0031-9201. doi: https://doi.org/10.1016/j.pepi.2006.03.018. URL https://www.sciencedirect.com/science/article/pii/S0031920106001038.
  27. [27]Dan Givoli. A tutorial on the adjoint method for inverse problems. Computer Methods in Applied Mechanics and Engineering, 380:113810, July 2021. ISSN 0045-7825. doi: 10.1016/j.cma.2021.113810. URL https://www.sciencedirect.com/science/article/pii/S0045782521001468.
  28. [28]Andreas Griewank. Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation. Optimization Methods and Software, 1(1):35–54, January 1992. ISSN 1055-6788, 1029-4937. doi: 10.1080/10556789208805505. URL https://www.tandfonline.com/doi/full/10.1080/10556789208805505.DOI
  29. [29]Andreas Griewank and Andrea Walther. Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation. ACM Transactions on Mathematical Software, 26(1):19–45, March 2000. ISSN 0098-3500, 1557-7295. doi: 10.1145/347837.347846. URL https://dl.acm.org/doi/10.1145/347837.347846.
  30. [30]William W. Symes. Reverse time migration with optimal checkpointing. GEOPHYSICS, 72(5):SM213–SM221, September 2007. ISSN 0016-8033, 1942-2156. doi: 10.1190/1.2742686. URL https://library.seg.org/doi/10.1190/1.2742686.
  31. [31]John E. Anderson, Lijian Tan, and Don Wang. Time-reversal checkpointing methods for RTM and FWI. GEOPHYSICS, 77(4):S93–S103, July 2012. ISSN 0016-8033, 1942-2156. doi: 10.1190/geo2011-0114.1. URL https://library.seg.org/doi/10.1190/geo2011-0114.1.DOI
  32. [32]Christian Boehm, Mauricio Hanzich, Josep de la Puente, and Andreas Fichtner. Wavefield compression for adjoint methods in full-waveform inversion. GEOPHYSICS, 81(6):R385–R397, 2016. doi: 10.1190/geo2015–0653.1. URL https://doi.org/10.1190/geo2015–0653.1.
  33. [33]Curt Da Silva, Yiming Zhang, Rajiv Kumar, and Felix J. Herrmann. Applications of low-rank compressed seismic data to full-waveform inversion and extended image volumes. GEOPHYSICS, 84(3):R371–R383, 2019. doi: 10.1190/geo2018–0116.1. URL https://doi.org/10.1190/geo2018–0116.1.
  34. [34]Samuel Omlin and Ludovic Räss. High-performance XPU Stencil Computations in Julia. JuliaCon Proceedings, 6(64):138, October 2024. ISSN 2642-4029. doi: 10.21105/jcon.00138. URL https://proceedings.juliacon.org/papers/10.21105/jcon.00138.DOI
  35. [35]Samuel Omlin, Ludovic Räss, and Ivan Utkin. Distributed Parallelization of xPU Stencil Computations in Julia. JuliaCon Proceedings, 6(65):137, November 2024. ISSN 2642-4029. doi: 10.21105/jcon.00137. URL https://proceedings.juliacon.org/papers/10.21105/jcon.00137.DOI
  36. [36]Giacomo Aloisi, Andrea Zunino, and Andreas Fichtner. SeismicWaves.jl: an efficient yet user-friendly Julia package for Full-Waveform Inversion on multi-xPUs. Technical Report EGU24–10916, Copernicus Meetings, March 2024. URL https://meetingorganizer.copernicus.org/EGU24/EGU24–10916.html.
  37. [37]Robert G. Clapp. Reverse time migration with random boundaries, pages 2809–2813. 2009. doi: 10.1190/1.3255432. URL https://library.seg.org/doi/abs/10.1190/1.3255432.DOI
  38. [38]Xukai Shen and Robert Clapp. Random boundary condition for memory-efficient waveform inversion gradient computation. GEOPHYSICS, 80:R351–R359, 11 2015. doi: 10.1190/geo2014–0542.1.DOI
  39. [39]Albert Tarantola. Inversion of seismic reflection data in the acoustic approximation. GEOPHYSICS, 49(8):1259–1266, August 1984. ISSN 0016-8033, 1942-2156. doi: 10.1190/1.1441754. URL https://library.seg.org/doi/10.1190/1.1441754.
  40. [40]Andreas Fichtner. Full Seismic Waveform Modelling and Inversion. Advances in Geophysical and Environmental Mechanics and Mathematics. Springer Berlin Heidelberg, 2011. ISBN 978-3-642-15806-3. doi: 10.1007/978-3-642-15807-0.DOI
  41. [41]Eddie Wadbro and Martin Berggren. Topology optimization of an acoustic horn. Computer Methods in Applied Mechanics and Engineering, 196(1-3):420–436, December 2006. ISSN 00457825. doi: 10.1016/j.cma.2006.05.005. URL https://linkinghub.elsevier.com/retrieve/pii/S0045782506001745.DOI
  42. [42]Jin Woo Lee and Yoon Young Kim. Rigid body modeling issue in acoustical topology optimization. Computer Methods in Applied Mechanics and Engineering, 198(9-12):1017–1030, February 2009. ISSN 00457825. doi: 10.1016/j.cma.2008.11.008. URL https://linkinghub.elsevier.com/retrieve/pii/S004578250800412X.DOI
  43. [43]Jianbin Du and Niels Olhoff. Minimization of sound radiation from vibrating bi-material structures using topology optimization. Structural and Multidisciplinary Optimization, 33(4-5):305–321, February 2007. ISSN 1615-147X, 1615-1488. doi: 10.1007/s00158-006-0088-9. URL http://link.springer.com/10.1007/s00158-006-0088-9.
  44. [44]Gil Ho Yoon, Jakob Søndergaard Jensen, and Ole Sigmund. Topology optimization of acoustic-structure interaction problems using a mixed finite element formulation. International Journal for Numerical Methods in Engineering, 70(9):1049–1075, May 2007. ISSN 0029-5981, 1097-0207. doi: 10.1002/nme.1900. URL https://onlinelibrary.wiley.com/doi/10.1002/nme.1900.
  45. [45]Maria B. Dühring, Jakob S. Jensen, and Ole Sigmund. Acoustic design by topology optimization. Journal of Sound and Vibration, 317(3-5):557–575, November 2008. ISSN 0022460X. doi: 10.1016/j.jsv.2008.03.042. URL https://linkinghub.elsevier.com/retrieve/pii/S0022460X08002812.DOI
  46. [46]Junghwan Kook, Kunmo Koo, Jaeyub Hyun, Jakob S. Jensen, and Semyung Wang. Acoustical topology optimization for Zwicker’s loudness model – Application to noise barriers. Computer Methods in Applied Mechanics and Engineering, 237-240:130–151, September 2012. ISSN 00457825. doi: 10.1016/j.cma.2012.05.004. URL https://linkinghub.elsevier.com/retrieve/pii/S0045782512001521.
  47. [47]J.S. Jensen and O. Sigmund. Topology optimization for nano-photonics. Laser & Photonics Reviews, 5(2):308–321, March 2011. ISSN 1863-8880, 1863-8899. doi: 10.1002/lpor.201000014. URL https://onlinelibrary.wiley.com/doi/10.1002/lpor.201000014.DOI
  48. [48]Rasmus E. Christiansen and Ole Sigmund. Inverse design in photonics by topology optimization: tutorial. Journal of the Optical Society of America B, 38(2):496, February 2021. ISSN 0740-3224, 1520-8540. doi: 10.1364/JOSAB.406048. URL https://opg.optica.org/abstract.cfm?URI=josab-38-2-496.DOI
  49. [49]Rasmus E. Christiansen and Ole Sigmund. Compact 2D line MATLAB code for inverse design in photonics by topology optimization: tutorial. Journal of the Optical Society of America B, 38(2):510, February 2021. ISSN 0740-3224, 1520-8540. doi: 10.1364/JOSAB.405955. URL https://opg.optica.org/abstract.cfm?URI=josab-38-2-510.
  50. [50]John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable parallel programming with CUDA. In ACM SIGGRAPH 2008 classes, pages 1–14, Los Angeles California, August 2008. ACM. ISBN 978-1-4503-7845-1. doi: 10.1145/1401132.1401152. URL https://dl.acm.org/doi/10.1145/1401132.1401152.DOI
  51. [51]Leon Herrmann, Tim Bürchner, László Kudela, and Stefan Kollmannsberger. A Memory Efficient Adjoint Method to Enable Billion Parameter Optimization on a Single GPU in Dynamic Problems [Software], September 2025. URL https://zenodo.org/doi/10.5281/zenodo.17157434.
  52. [52]Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs], January 2017. URL http://arxiv.org/abs/1412.6980.
  53. [53]Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1):503–528, August 1989. ISSN 1436-4646. doi: 10.1007/BF01589116. URL https://doi.org/10.1007/BF01589116.
  54. [54]Krister Svanberg. The method of moving asymptotes—a new method for structural optimization. International Journal for Numerical Methods in Engineering, 24(2):359–373, February 1987. ISSN 0029-5981, 1097-0207. doi: 10.1002/nme.1620240207. URL https://onlinelibrary.wiley.com/doi/10.1002/nme.1620240207.
  55. [55]Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Ev- geni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. Van Der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, António H. Ribeiro, Fabian Pedregosa, Paul Van Mulbregt, SciPy 1.0 Contributors, Aditya Vijayakumar, Alessandro Pietro Bardelli, Alex Rothberg, Andreas Hilboll, Andreas Kloeckner, Anthony Scopatz, Antony Lee, Ariel Rokem, C. Nathan Woods, Chad Fulton, Charles Masson, Christian Häggström, Clark Fitzgerald, David A. Nicholson, David R. Hagen, Dmitrii V. Pasechnik, Emmanuele Olivetti, Eric Martin, Eric Wieser, Fabrice Silva, Felix Lenders, Florian Wilhelm, G. Young, Gavin A. Price, Gert-Ludwig Ingold, Gregory E. Allen, Gregory R. Lee, Hervé Audren, Irvin Probst, Jörg P. Dietrich, Jacob Silberra, James T Webber, Janko Slavic, Joel Nothman, Johannes Buchner, Johannes Kulick, Johannes L. Schönberger, José Vinícius De Miranda Cardoso, Joscha Reimner, Joseph Harrington, Juan Luis Cano Rodríguez, Juan Nunez-Iglesias, Justin Kuczynski, Kevin Tritz, Martin Thoma, Matthew Newville, Matthias Kümmerer, Maximilian Bolingbroke, Michael Tarpine, Mikhail Pak, Nathaniel J. Smith, Nikolai Nowaczyk, Nikolay Shebanov, Oleksandr Pavlyk, Per A. Brodtkorb, Perry Lee, Robert T. McGibbon, Roman Feldbauer, Sam Lewis, Sam Tygier, Scott Sievert, Sebastiano Vigna, Stefan Peterson, Surhud More, Tadeusz Pudlik, Takuya Oshima, Thomas J. Pingel, Thomas P. Robitaille, Thomas Spura, Thouis R. Jones, Tim Cera, Tim Leslie, Tiziano Zito, Tom Krauss, Utkarsh Upadhyay, Yaroslav O. Halchenko, and Yoshiki Váquez-Baeza. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 17(3):261–272, March 2020. ISSN 1548-7091, 1548-7105. doi: 10.1038/s41592-019-0686-2. URL https://www.nature.com/articles/s41592-019-0686-2.DOI
  56. [56]Tim Büch­ner, Philipp Kopp, Stefan Kollmannsberger, and Ernst Rank. Immersed boundary parametrizations for full waveform inversion. Computer Methods in Applied Mechanics and Engineering, 406:115893, March 2023. ISSN 0045-7825. doi: 10.1016/j.cma.2023.115893. URL https://www.sciencedirect.com/science/article/pii/S0045782523000166.
  57. [57]A. Fichtner, H. P. Bunge, and H. Igel. The adjoint method in seismology: I. Theory. Physics of the Earth and Planetary Interiors, 157(1):86–104, August 2006. ISSN 0031-9201. doi: 10.1016/j.pepi.2006.03.016. URL https://www.sciencedirect.com/science/article/pii/S0031920106001051.DOI
  58. [58]A. Fichtner, H. P. Bunge, and H. Igel. The adjoint method in seismology—: II. Applications: traveltimes and sensitivity functionals. Physics of the Earth and Planetary Interiors, 157(1):105–123, August 2006. ISSN 0031-9201. doi: 10.1016/j.pepi.2006.03.018. URL https://www.sciencedirect.com/science/article/pii/S0031920106001038.DOI
  59. [59]Tim Büchner, Philipp Kopp, Stefan Kollmannsberger, and Ernst Rank. Isogeometric multi-resolution full waveform inversion based on the finite cell method. Computer Methods in Applied Mechanics and Engineering, 417:116286, December 2023. doi: 10.1016/j.cma.2023.116286. URL https://www.sciencedirect.com/science/article/pii/S0045782523004103.
  60. [60]Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models. Acta Numerica, 28:1–174, May 2019. ISSN 0962-4929, 1474-0508. doi: 10.1017/S0962492919000059. URL https://www.cambridge.org/core/journals/acta-numerica/article/solving-inverse-problems-using-datadriven-models/CE5B3725869AEAF46E04874115B0AB15.DOI
  61. [61]J. K. Guest, J. H. Prévost, and T. Belytschko. Achieving minimum length scale in topology optimization using nodal design variables and projection functions. International Journal for Numerical Methods in Engineering, 61(2):238–254, September 2004. ISSN 0029-5981, 1097-0207. doi: 10.1002/nme.1064. URL https://onlinelibrary.wiley.com/doi/10.1002/nme.1064.DOI
  62. [62]Fengwen Wang, Boyan Stefanov Lazarov, and Ole Sigmund. On projection methods, convergence and robust formulations in topology optimization. Structural and Multidisciplinary Optimization, 43(6):767–784, June 2011. ISSN 1615-1488. doi: 10.1007/s00158-010-0602-y. URL https://doi.org/10.1007/s00158-010-0602-y.
  63. [63]Lei Li and Kapil Khandelwal. Volume preserving projection filters and continuation methods in topology optimization. Engineering Structures, 85:144–161, February 2015. ISSN 0141-0296. doi: 10.1016/j.engstruct.2014.10.052. URL https://www.sciencedirect.com/science/article/pii/S01410296140065679.DOI
  64. [64]M. P. Bendsøe. Optimal shape design as a material distribution problem. Structural optimization, 1(4):193–202, December 1989. ISSN 1615-1488. doi: 10.1007/BF01650949. URL https://doi.org/10.1007/BF01650949.
  65. [65]M. P. Bendsøe and O. Sigmund. Material interpolation schemes in topology optimization. Archive of Applied Mechanics, 69(9):635–654, November 1999. ISSN 1432-0681. doi: 10.1007/s004190050248. URL https://doi.org/10.1007/s004190050248.
  66. [66]Martin P. Bendsøe and O. Sigmund. Topology optimization: theory, methods, and applications. Springer, Berlin ; New York, 2003. ISBN 978-3-540-42992-0.
  67. [67]Jaeyub Hyun and H. Alicia Kim. Transient level-set topology optimization of a planar acoustic lens working with short-duration pulse. The Journal of the Acoustical Society of America, 149(5):3010–3026, May 2021. ISSN 0001-4966, 1520-8524. doi: 10.1121/10.0004819. URL https://pubs.aip.org/jasa/article/149/5/3010/607500/Transient-level-set-topology-optimization-of-a.DOI
  68. [68]Cetin B. Dilgen and Niels Aage. Topology optimization of transient vibroacoustic problems for broadband filter design using cut elements. Finite Elements in Analysis and Design, 234:104123, July 2024. ISSN 0168-874X. doi: 10.1016/j.finel.2024.104123. URL https://www.sciencedirect.com/science/article/pii/S0168874X24000179.DOI
  69. [69]Adrien Pelat, François Gautier, Stephen C. Conlon, and Fabio Semperlotti. The acoustic black hole: A review of theory and applications. Journal of Sound and Vibration, 476:115316, June 2020. ISSN 0022-460X. doi: 10.1016/j.jsv.2020.115316. URL https://www.sciencedirect.com/science/article/pii/S0022460X20301474.DOI
  70. [70]Abbas Mousavi, Martin Berggren, Linus Hägg, and Eddie Wadbro. Topology optimization of a waveguide acoustic black hole for enhanced wave focusing. The Journal of the Acoustical Society of America, 155(1):742–756, January 2024. ISSN 1520-8524. doi: 10.1121/10.0024470.DOI
  71. [71]Leon Herrmann, Ole Sigmund, Viola Muning Li, Christian Vogl, and Stefan Kollmannsberger. On neural networks for generating better local optima in topology optimization. Structural and Multidisciplinary Optimization, 67(11):192, November 2024. ISSN 1615-1488. doi: 10.1007/s00158-024-03908-6. URL https://doi.org/10.1007/s00158-024-03908-6.
  72. [72]O. Sigmund and J. Petersson. Numerical instabilities in topology optimization: A survey on procedures dealing with checkerboards, mesh-dependencies and local minima. Structural optimization, 16(1):68–75, August 1998. ISSN 1615-1488. doi: 10.1007/BF01214002. URL https://doi.org/10.1007/BF01214002.
  73. [73]Tyler E. Bruns and Daniel A. Tortorelli. Topology optimization of non-linear elastic structures and compliant mechanisms. Computer Methods in Applied Mechanics and Engineering, 190(26):3443–3459, March 2001. ISSN 0045-7825. doi: 10.1016/S0045-7825(00)00278-4. URL https://www.sciencedirect.com/science/article/pii/S0045782500002784.DOI
  74. [74]Blaise Bourdin. Filters in topology optimization. International Journal for Numerical Methods in Engineering, 50(9):2143–2158, March 2001. ISSN 0029-5981, 1097-0207. doi: 10.1002/nme.116. URL https://onlinelibrary.wiley.com/doi/10.1002/nme.116.DOI
  75. [75]Niels Aage, Erik Andreassen, Boyan S. Lazarov, and Ole Sigmund. Giga-voxel computational morphogenesis for structural design. Nature, 550(7674):84–86, October 2017. ISSN 1476-4687. doi: 10.1038/nature23911. URL https://www.nature.com/articles/nature23911.DOI
  76. [76]Rasmus E. Christiansen, Boyan S. Lazarov, Jakob S. Jensen, and Ole Sigmund. Creating geometrically robust designs for highly sensitive problems using topology optimization: Acoustic cavity design. Structural and Multidisciplinary Optimization, 52(4):737–754, October 2015. ISSN 1615-147X, 1615-1488. doi: 10.1007/s00158-015-1265-5. URL http://link.springer.com/10.1007/s00158-015-1265-5.DOI
  77. [77]Ole Sigmund. Morphology-based black and white filters for topology optimization. Structural and Multidisciplinary Optimization, 33(4):401–424, April 2007. ISSN 1615-1488. doi: 10.1007/s00158-006-0087-x. URL https://doi.org/10.1007/s00158-006-0087-x.
  78. [78]Ole Sigmund. Manufacturing tolerant topology optimization. Acta Mechanica Sinica, 25(2):227–239, April 2009. ISSN 1614-3116. doi: 10.1007/s10409-009-0240-z. URL https://doi.org/10.1007/s10409-009-0240-z.
  79. [79]Hans Petter Langtangen and Svein Linge. Finite Difference Computing with PDEs: A Modern Software Approach, volume 16 of Texts in Computational Science and Engineering. Springer International Publishing, Cham, 2017. ISBN 978-3-319-55455-6 978-3-319-55456-3. doi: 10.1007/978-3-319-55456-3. URL http://link.springer.com/10.1007/978-3-319-55456-3.DOI
  80. [80]Leon Herrmann, Tim B̈uchner, Felix Dietrich, and Stefan Kollmannsberger. On the Use of Neural Networks for Full Waveform Inversion [Software], August 2024. URL https://data.mendeley.com/datasets/7kps2hnj6g/2.
  81. [81]Armand Wirgin. The inverse crime, January 2004. URL http://arxiv.org/abs/math-ph/0401050. arXiv:math-ph/0401050.
  82. [82]Andreas Fichtner. Lecture notes on inverse theory, 07 2021. This content is a preprint and has not been peer-reviewed.DOI
  83. [83]L. Hug, M. Potten, G. Stockinger, K. Thuro, and S. Kollmannsberger. A three-field phase-field model for mixed-mode fracture in rock based on experimental determination of the mode II fracture toughness. Engineering with Computers, 38(6):5563–5581, December 2022. ISSN 0177-0667, 1435-5663. doi: 10.1007/s00366-022-01684-9. URL https://link.springer.com/10.1007/s00366-022-01684-9.DOI
  84. [84]Jamshid Parvizian, Alexander D̈uster, and Ernst Rank. Finite cell method. Computational Mechanics, 41(1):121–133, December 2007. ISSN 1432-0924. doi: 10.1007/s00466-007-0173-y. URL https://doi.org/10.1007/s00466-007-0173-y.
  85. [85]Alexander D̈uster, Ernst Rank, and Barna Szabó. The Finite Element and Finite Cell Methods. In Erwin Stein, René Borst, and Thomas J R Hughes, editors, Encyclopedia of Computational Mechanics Second Edition, pages 1–35. Wiley, 1 edition, December 2017. ISBN 978-1-119-00379-3 978-1-119-17681-7. doi: 10.1002/9781119176817.ecm2003g. URL https://onlinelibrary.wiley.com/doi/10.1002/9781119176817.ecm2003g.DOI
  86. [86]O. Sigmund. A 99 line topology optimization code written in Matlab. Structural and Multidisciplinary Optimization, 21(2):120–127, April 2001. ISSN 1615-1488. doi: 10.1007/s001580050176. URL https://doi.org/10.1007/s001580050176.
  87. [87]Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B. Shah. Julia: A Fresh Approach to Numerical Computing. SIAM Review, 59(1):65–98, January 2017. ISSN 0036-1445, 1095-7200. doi: 10.1137/141000671. URL https://epubs.siam.org/doi/10.1137/141000671.DOI
  88. [88]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.arxiv.org/abs/1706.03762
  89. [89]Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhanh, and Pablo Villalobos. Compute Trends Across Three Eras of Machine Learning. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8, July 2022. doi: 10.1109/IJCNN55064.2022.9891914. URL https://ieeexplore.ieee.org/document/9891914. ISSN: 2161-4407.DOI
  90. [90]Kailai Xu and Eric Darve. The Neural Network Approach to Inverse Problems in Differential Equations, January 2019. URL http://arxiv.org/abs/1901.07758.
  91. [91]Jens Berg and Kaj Nyström. Neural networks as smooth priors for inverse problems for PDEs. Journal of Computational Mathematics and Data Science, 1:100008, September 2021. ISSN 27724158. doi: 10.1016/j.jc mds.2021.100008. URL https://linkinghub.elsevier.com/retrieve/pii/S2772415821000043.DOI
  92. [92]Weiqiang Zhu, Kailai Xu, Eric Darve, Biondo Biondi, and Gregory C. Beroza. Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification. GEOPHYSICS, 87(1):R93–R109, January 2022. ISSN 0016-8033, 1942-2156. doi: 10.1190/geo2020-0933.1. URL https://library.seg.org/doi/10.1190/geo2020-0933.1.DOI
  93. [93]Peng Jiang, Qingyang Wang, Yuxiao Ren, Senlin Yang, and Ningbo Li. Full waveform inversion based on inversion network reparameterized velocity. Geophysical Prospecting, 72(1):52–67, January 2024. ISSN 0016-8025, 1365-2478. doi: 10.1111/1365-2478.13292. URL https://onlinelibrary.wiley.com/doi/10.1111/1365-2478.13292.DOI
  94. [94]Divya Shyam Singh, Leon Herrmann, Qing Sun, Tim Bürchner, Felix Dietrich, and Stefan Kollmannsberger. Accelerating full waveform inversion by transfer learning. Computational Mechanics, February 2025. ISSN 1432-0924. doi: 10.1007/s00466-025-02600-w. URL https://doi.org/10.1007/s00466-025-02600-w.
  95. [95]Feng Liu, Yaxing Li, Rui Su, Jianping Huang, and Lei Bai. Deep Reparameterization for Full Waveform Inversion: Architecture Benchmarking, Robust Inversion, and Multiphysics Extension, April 2025. URL http://arxiv.org/abs/2504.17375. arXiv:2504.17375.
  96. [96]Lucas Norder, Shunyu Yin, Matthijs H. J. de Jong, Francesco Stallone, Hande Aydogmus, Paolo M. Sberna, Miguel A. Bessa, and Richard A. Norte. Pentagonal photonic crystal mirrors: scalable lightsails with enhanced acceleration via neural topology optimization. Nature Communications, 16(1):2753, March 2025. ISSN 2041-1723. doi: 10.1038/s41467-025-57749-y. URL https://www.nature.com/articles/s41467-025-57749-y.DOI
  97. [97]Stephan Hoyer, Jascha Sohl-Dickstein, and Sam Greydanus. Neural reparameterization improves structural optimization, September 2019. URL http://arxiv.org/abs/1909.04240.
  98. [98]Hao Deng and Albert C. To. Topology optimization based on deep representation learning (DRL) for compliance and stress-constrained design. Computational Mechanics, 66(2):449–469, August 2020. ISSN 1432-0924. doi: 10.1007/s00466-020-01859-5. URL https://doi.org/10.1007/s00466-020-01859-5.
  99. [99]Liang Chen and Mo-How Herman Shen. A New Topology Optimization Approach by Physics-Informed Deep Learning Process. Advances in Science, Technology and Engineering Systems Journal, 6(4):233–240, July 2021. ISSN 24156698, 24156698. doi: 10.25046/aj0604277. URL https://astesj.com/v06/i04/p27/.DOI
  100. [100]Aaditya Chandrasekhar and Krishnan Suresh. TOuNN: Topology Optimization using Neural Networks. Structural and Multidisciplinary Optimization, 63(3):1135–1149, March 2021. ISSN 1615-147X, 1615-1488. doi: 10.1007/s00158-020-02748-4. URL http://link.springer.com/10.1007/s00158-020-02748-4.DOI
  101. [101]Aaditya Chandrasekhar and Krishnan Suresh. Multi-Material Topology Optimization Using Neural Networks. Computer-Aided Design, 136:103017, July 2021. ISSN 0010-4485. doi: 10.1016/j.cad.2021.103017. URL https://www.sciencedirect.com/science/article/pii/S0010448521000282.DOI
  102. [102]Aaditya Chandrasekhar and Krishnan Suresh. Approximate Length Scale Filter in Topology Optimization using Fourier Enhanced Neural Networks. Computer-Aided Design, 150:102377, September 2022. ISSN 0010-4485. doi: 10.1016/j.cad.2022.103277. URL https://www.sciencedirect.com/science/article/pii/S0010448522000574.DOI
  103. [103]Connor N. W. Mallon, Aaron W. Thornton, Matthew R. Hill, and Santiago Badia. Neural Level Set Topology Optimization Using Unfitted Finite Elements, February 2024. URL http://arxiv.org/abs/2303.13672.
  104. [104]Suryanarayanan Manoj Sanu, Alejandro M. Aragon, and Miguel A. Bessa. Neural topology optimization: the good, the bad, and the ugly, July 2024. URL http://arxiv.org/abs/2407.13954.
  105. [105]Igor Kuszczak, Gawel Kus, Federico Bosi, and Miguel A. Bessa. Meta-neural Topology Optimization: Knowledge Infusion with Meta-learning, February 2025. URL http://arxiv.org/abs/2502.01830. arXiv:2502.01830.

Paper details

vlnc:W_ch9aec8b
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents