betaFeedback

WorksComputer Science

Assessing AI-Enhanced Single-Sweep Approximations for Problems with Forward-Peaked Scattering in Slab Geometry

Abstract

While the Boltzmann transport equation can accurately model transport problems with highly forward-peaked scattering, obtaining its solution can become arbitrarily slow due to near-unity spectral radius associated with source iteration. Standard acceleration techniques like diffusion synthetic acceleration and nonlinear diffusion acceleration obtain merely one order of magnitude speedups compared to source iteration due to slowly decaying error moments. Additionally, converging approximations to the Boltzmann equation like Fokker-Planck and Boltzmann Fokker Planck run into similar problems with slow convergence. In this paper we assess the feasibility of using Fourier neural operators to obtain AI-enhanced low order, and single-sweep solutions for the transport equation in slab geometry using a predictor-corrector framework.

INTRODUCTION

Forward-peaked transport finds application in several important research areas including radiation oncology physics, plasma physics, astrophysics, shielding, etc. Such problems are characterized by small mean free paths with nearly singular differential scattering cross-sections in the forward direction and require relatively large expansions to represent scattering adequately. Several approximations, including Fokker-Planck (FP) [1], generalized Fokker Planck [2], Boltzmann Fokker Planck (BFP) [3], and generalized Boltzmann Fokker Planck (GBFP) [4] have been developed to address this problem by manipulating the scattering term of the Boltzmann transport equation (BTE). Additionally, acceleration techniques like diffusion synthetic acceleration (DSA) [5], nonlinear diffusion acceleration (NDA) [6], transport synthetic acceleration (TSA) [7], Angular multigrid (AMG) [8], Fokker Planck synthetic acceleration (FPSA) [9], nonlinear and modified Fokker Planck acceleration (MFPA) [10], modified PL acceleration (MPA) [11], quasi diffusion [12], etc. have also been investigated to speed up convergence for forward-peaked transport problems. Standard methods like DSA and NDA do not speed up calculations as effectively as they do for isotropic and weakly anisotropic problems because these methods do not attenuate higher order error moments [9]. Other methods like FPSA and AMG that attenuate higher order error moments have been demonstrated to be effective in 1D, but such methods are highly problem dependent [8, 9]. To our knowledge, no single acceleration or approximation approach has been shown to work flawlessly in all transport regimes. In this paper, we explore using neural operators to approximate converged solutions without having to go through an exhaustive iteration process. Preliminary studies involving use of Fourier neural operator neural network (FNO) to approximate high order fine mesh transport solutions has been summarized in [13, 14]. In this paper, we investigate predictor-corrector frameworks to estimate higher fidelity solutions using lower order approximations. Predictors employed in this paper include S2S_2 discrete ordinates solutions, along with single-sweep estimates associated with source iteration, and pure absorbers. To our knowledge, this is the first time operator learning is being explored to develop single-sweep approximations to transport solutions.

We primarily work with the slab geometry transport equation and begin by reviewing it. With angular flux ψ\psi, lthl^{\mathrm{th}} Legendre moment of angular flux ϕl\phi_l, total macroscopic cross-section σt\sigma_t, lthl^{\mathrm{th}} order Legendre polynomial PlP_l, scattering cross-section moments σs,l\sigma_{s,l}, and an internal source QQ, we consider the following slab geometry transport equation [15]:

μddxψ(x,μ)+σt(x)ψ(x,μ)=∑l=0L2l+12Pl(μ)σs,l(x)ϕl(x)+Q(x,μ).(1)\mu\frac{d}{dx}\psi(x,\mu) + \sigma_t(x)\psi(x,\mu) = \sum_{l=0}^{L} \frac{2l+1}{2} P_l(\mu)\sigma_{s,l}(x)\phi_l(x) + Q(x,\mu). \tag*{(1)}

with moments of the angular flux are determined using the following relation:

ϕl(x)=∫−11ψ(x,μ)Pl(μ) dμ.(2)\phi_l(x) = \int_{-1}^{1} \psi(x,\mu)P_l(\mu)\,d\mu. \tag*{(2)}

The following sections will introduce the predictor-corrector approach. We provide a brief overview of FNO the next section, along with some notes on how to use them to speed up transport calculations and conclude in § 3.

NEURAL OPERATOR BASED APPROXIMATIONS

The main goal of operator learning is to find properties of systems and equations from data [16]. Traditionally, neural networks have generally been used to describe mappings from one vector to another. This is limiting because these mappings lack discrete-continuous consistency. Recent neural operator architectures overcome this drawback by inferring operators that map input to output functions. Several different types of neural operator architectures have been explored in the literature [16, 17]. In this paper, we focus on the FNO framework [17].

Fourier Neural Operator Neural Network

While a detailed explanation of an FNO can be found in [17] we briefly outline noteworthy details and implementation steps next. With two functions aa and uu of independent variables xx on domain DD, and a mapping GG such that

u(x)=G(a(x)),(3)u(x) = G(a(x)), \tag*{(3)}

the goal is to determine an approximation Gθ≈GG_\theta\approx G with learnable parameters θ\theta, where,

Gθ=G0∘G1⋯∘GK,(4)G_\theta= G_0 \circ G_1 \cdots\circ G_K, \tag*{(4)}

is a neural network composed of network layers GkG_k represented according to

Gkv(x)=σ(Bk+∫DK(x,y)v(y) dy).(5)G_k v(x) = \sigma\left(B_k + \int_D K(x,y)v(y)\,dy\right). \tag*{(5)}

Here, σ\sigma, BkB_k, K(x,y)K(x,y), and v(x)v(x) represent the activation function, bias function, convolution kernel, and an intermediate network function. With NN neurons per layer, this representation requires roughly O(N2)O(N^2) operations for evaluation. Li et al. [17] introduced the idea of using Fourier transforms to convert the convolution into a simple product in Fourier space and reduced the operational overhead to O(Nlog⁡N)O(N\log N). FNOs consist of four main components – the lifting layer, Fourier layers, nonlinear activation, and projection layer. The lifting layer projects input functions into a higher dimensional feature space by linear transformation PP

v0(x)=P(x).(6)v_0(x) = P(x). \tag*{(6)}

Fourier layers apply Fourier transforms according to Eq. 6a, then multiply by learnable weights WθW_\theta in the Fourier space as shown in Eq. 6b, and apply inverse Fourier transforms and add in bias and residual connections according to Eq. 6c [17].

v^j(k)=F(vj)(k).(7)\widehat{v}_j(k) = F(v_j)(k). \tag*{(7)}
v^j′(k)=Wθv^j(k)(8)\widehat{v}'_j(k) = W_\theta\widehat{v}_j(k) \tag*{(8)}
vj+1(x)=F−1(v^j′)(x)+vj(x)(9)v_{j+1}(x) = F^{-1}(\widehat{v}'_j)(x) + v_j(x) \tag*{(9)}

Further, a nonlinearity is introduced by pointwise applying a nonlinear activation function σ\sigma according to

vj+1(x)=σ(vj+1(x)).(10)v_{j+1}(x) = \sigma\left(v_{j+1}(x)\right). \tag*{(10)}

The solution is obtained by projecting output onto the function space using projection operator QQ

u(x)=Q(vK)(x).(11)u(x) = Q(v_K)(x). \tag*{(11)}

A pictorial representation of this setup is presented in Figure 1 [17]

Fourier Neural Operator Neural Network Framework

Figure 1. Fourier Neural Operator Neural Network Framework.

Learnable parameters such as lifting, projection, weighting, and bias functions can be inferred by minimizing the loss function

ϵθ=1N∑iN∣ui,FNO−ui∣2,(12)\epsilon_{\theta} = \frac{1}{N}\sum_{i}^{N}\left\lvert u_{i,FNO}-u_i\right\rvert^2, \tag*{(12)}

using standard optimizers [18].

Predictor-Corrector Approximation Framework

We employ standard predictor-corrector setup to refine lower fidelity solutions to obtain higher fidelity estimates. In this section we consider three types of predictors – single sweep SN_N, pure absorber, and fully converged S2_2 solutions. The predicted solutions are then passed on to the FNO architecture to implicitly correct (map) predicted approximations to higher fidelity estimates. We begin by introducing some preliminaries. Rewriting Eq. (1) in operator notation,

Lψ(x,μ)=Sψ(x,μ)+Q(x,μ),(13)L\psi(x,\mu)=S\psi(x,\mu)+Q(x,\mu), \tag*{(13)}

with

L=μddx+σt(x),(14)L=\mu\frac{d}{dx}+\sigma_t(x), \tag*{(14)}
S=∑l=0L(2l+1)2Pl(μ)σs,l(x)ϕl(x),(15)S=\sum_{l=0}^{L}\frac{(2l+1)}{2}P_l(\mu)\sigma_{s,l}(x)\phi_l(x), \tag*{(15)}

the discrete ordinates formulation with angle index nn takes the following form [13]

Lψ(x,μn)=Sψ(x,μn)+Q(x,μn).(16)L\psi(x,\mu_n)=S\psi(x,\mu_n)+Q(x,\mu_n). \tag*{(16)}

With iteration index m\mathcal{m}, source iteration for SN equations takes the following form

Lψm+1(x,μn)=Sψm(x,μn)+Q(x,μn).(17)L\psi^{m+1}(x,\mu_n)=S\psi^m(x,\mu_n)+Q(x,\mu_n). \tag*{(17)}

The predictor-corrector approximation procedure breaks solution into two parts. Obtaining a low order (LO) approximation

and correcting it to approximate higher order (HO) solution:

Predict: ϕ0,LO,(18)\mathrm{Predict:}\ \phi_{0,LO}, \tag*{(18)}
Correct: G(ϕ0,LO)≈ϕ0,HO(19)\mathrm{Correct:}\ G(\phi_{0,LO})\approx\phi_{0,HO} \tag*{(19)}

This is similar to the predict-correct-iterate framework [9] employed for synthetic acceleration, except the iterate part is omitted. We train FNOs with 4 Fourier layers containing 64 Fourier modes each, along with the ReLU activation function [19] using the JAX framework [20]. Each case has four in-channels and one out-channel. While the input function varies depending on the choice of the function being mapped to the higher fidelity solution, each scenario additionally takes in the anisotropy factor and scattering ratio as inputs. The ADAM optimizer was used to minimize training losses [21] with learning rate of 1×10−41\times10^{-4} and batch size of 25. An average relative L2L_2 error norm was used to characterize the quality of inferred mappings.

FNO-enhanced single sweep solutions

The first predictor approach considered in this paper is the scalar flux obtained after one source iteration step (SS):

ψ1(x,μn)=L−1Sψ0(x,μn)+L−1Q(x,μn),(20)\psi^{1}(x,\mu_n) = L^{-1}S\psi^{0}(x,\mu_n) + L^{-1}Q(x,\mu_n), \tag*{(20)}

with higher fidelity approximation obtained using FNO such that

ϕ0,HO≈Gθ,SS(ϕ01).(21)\phi_{0,HO} \approx G_{\theta,SS}(\phi_{0}^{1}). \tag*{(21)}

FNO-enhanced pure absorber solutions

The second predictor method assumes a pure absorber (PA) system and ignores all scatter. The solution to such problems converges in one iteration and therefore this approach can also be deemed “single-sweep”. In this scenario, Eq. 11 takes the following form

ψ(x,μn)=L−1Q(x,μn).(22)\psi(x,\mu_n) = L^{-1}Q(x,\mu_n). \tag*{(22)}

The predictor ϕ0,PA\phi_{0,PA} obtained by solving Eq. 15a, and the correction is obtained by mapping this solution to higher fidelity solution using an FNO:

ϕ0,HO≈Gθ,PA(ϕ0,PA).(23)\phi_{0,HO} \approx G_{\theta,PA}(\phi_{0,PA}). \tag*{(23)}

We note that this approach becomes a special case of the SS approach when initial flux is set to zero in Eq. 14a. This,

however, is not the case here as we initialize the flux according to the beam source. Therefore, the two approaches return slightly different input fluxes.

FNO-enhanced S2 solutions

The final predictor approach considered in this paper converges the S2_2 solutions to desired tolerance according to

Lψm+1(x,μ1)=Sψm(x,μ1)+Q(x,μ1),(24)L\psi^{m+1}(x,\mu_1) = S\psi^{m}(x,\mu_1) + Q(x,\mu_1), \tag*{(24)}
Lψm+1(x,μ2)=Sψm(x,μ2)+Q(x,μ2),(25)L\psi^{m+1}(x,\mu_2) = S\psi^{m}(x,\mu_2) + Q(x,\mu_2), \tag*{(25)}

to obtain solution ϕ0,S2\phi_{0,S_2} and maps it back to the HO solution using an FNO such that

ϕ0,HO≈Gθ,S2(ϕ0,S2).(26)\phi_{0,HO} \approx G_{\theta,S_2}(\phi_{0,S_2}). \tag*{(26)}

Note that while the method to train FNOs Gθ,SSG_{\theta,SS}, Gθ,PAG_{\theta,PA}, and Gθ,S2G_{\theta,S_2} is the same, these are three distinct mappings. In other words, each predictor has a companion corrector mapping, and they must always be used in conjunction with each other.

Numerical Experiments

For preliminary assessment of FNO-enhanced solutions, we consider a slab geometry problem of size 50 cm with a beam source along the most forward direction. We incorporate anisotropy into the problem using the Henyey-Greenstein kernel, whose scattering moments are evaluated according to [22].

σs,lHGK=σsgl.(27)\sigma_{s,l}^{HGK} = \sigma_s g^l. \tag*{(27)}

As the anisotropy factor gg goes from zero to unity, scattering goes from isotropic to highly anisotropic and forward peaked. FNO correctors are trained using a dataset of 512 cases with random material properties. The transport equation is discretized using linear discontinuous (LD) finite element method and solved using the source iteration algorithm on a Precision 7920 workstation with two Intel Xeon Gold 6148 CPUs and 192 GB of RAM. While S16_{16}P15_{15} estimates represent HO solutions, LO solutions are obtained using predictor setups described in § 2.2. The anisotropy factor for the Henyey-Greenstein phase function is randomly varied from 0.8 to 1 and the dataset is divided into training and test sets in 3:1 proportion. An NVIDIA A100 GPU was used via Google Colab for training which took roughly 4 minutes for each of the predictor-corrector scenario. It is noteworthy that changing to an NVIDIA T4 GPU increased training runtime to roughly 14 minutes. Training and inference were found to be hardware dependent as expected. Figure 2, Figure 3, and Figure 4 present example cases for the three predictor-corrector setups. The scattering ratio and anisotropy factors for this case are 0.995 and 0.977, respectively. Similarly, a general runtime comparison is not possible because spectral radius is highly problem dependent.

Scalar flux comparison with SS predictor

Figure 2. Scalar flux comparison with SS predictor.

Scalar flux comparison with PA predictor

Figure 3. Scalar flux comparison with PA predictor.

Scalar flux comparison with $S_2$ predictor

Figure 4. Scalar flux comparison with S2S_2 predictor.

We observe from Figure 2, Figure 3, and Figure 4 that while stand-alone LO and single-sweep approximations fail to capture the physics, incorporating FNO-based corrections improves solution quality significantly for all three predictors. Our general observation was that if appropriate physics parameters are provided as input, and low order approximations capture overall variability in solutions, FNOs work quite well. Having said that, we also observed that not using physical parameters with input, degrades mapping for SS because scattering ratio and anisotropy factors often play a crucial role in determining how the first sweep relates to the converged solution.

With average time to generate predictor estimate t‾p\overline{t}_{\mathrm{p}}, that to obtain higher fidelity solution t‾HO\overline{t}_{\mathrm{HO}}, average time to obtain FNO-based corrections t‾c\overline{t}_{\mathrm{c}} and average relative L2L_2 error norms ϵ‾\overline{\epsilon}, we summarize average speedups and errors in Table 1. We observe, as expected, that FNO-based corrections take a fraction of the time required by source iteration to return higher fidelity solutions. Specifically, for the dataset used in this paper, obtaining FNO-based corrections was observed to be roughly 3-4 orders of magnitude faster than obtaining higher fidelity solutions by source iteration. Additionally, this is significantly larger than the two orders of magnitude speedup seen with evaluation of LO and single-sweep predictors. Therefore, we note that runtime speedups are dominated by the choice of LO predictors. For the cases under consideration, while we observe that converged S2S_2 solutions was the most efficient, the single-sweep methods roughly provided the same order of speedups. This pattern is not unexpected because S2S_2 solutions, with their coarse angular mesh converge relatively quickly and do not require as many iterations as higher fidelity solutions. Additionally, with their coarse mesh, computational overhead is also relatively smaller than that required while sweeping through finer meshes.

PredictorSSPAConverged S2S_2
t‾p\overline{t}_{\mathrm{p}}6.12×10−26.12 \times10^{-2}3.66×10−23.66 \times10^{-2}1.37×10−21.37 \times10^{-2}
t‾HO\overline{t}_{\mathrm{HO}}
t‾c\overline{t}_{\mathrm{c}} [s]4.80×10−44.80 \times10^{-4}4.46×10−44.46 \times10^{-4}4.53×10−44.53 \times10^{-4}
ϵ‾\overline{\epsilon}1.27×10−31.27 \times10^{-3}1.31×10−31.31 \times10^{-3}1.11×10−31.11 \times10^{-3}

Table 1. Average Speedups and Errors.

CONCLUSIONS

FNO-enhanced predictor-corrector setup was investigated using a small dataset of 512 samples. Three distinct predictor mechanisms were tested with all three methods proving to be sufficiently accurate and comparable in efficiency. In conclusion, we demonstrate FNO as a novel method for mapping lower fidelity predictions to higher fidelity solutions efficiently. Future work will focus on moving to full 3D transport solutions and investigating how to leverage FNO for preconditioning.

ACKNOWLEDGMENTS

We would like to thank Prof. John C. Lee from the University of Michigan for discussions and Prof. Dean Wang from The Ohio State University for his kind support.

References

  1. [1]G. C. Pomraning, “The Fokker-Planck operator as an asymptotic limit,” Mathematical Models and Methods in Applied Sciences, vol. 02, no. 1, pp. 21-36, 1992.
  2. [2]A. K. Prinja, and G. C. Pomraning, “A generalized Fokker-Planck model for transport of collimated beams,” Nuclear Science and Engineering, vol. 137, pp. 227-235, 2001.DOI
  3. [3]M. Landesman, and J. E. Morel, “Angular Fokker-Planck decomposition and representation techniques,” Nuclear Science and Engineering, vol. 103, 1-11, 1989.DOI
  4. [4]D. A. Dixon, “A computationally efficient moment-preserving Monte Carlo transport method with implementation in GEANT4,” PhD thesis, University of New Mexico, Albuquerque, NM, 2014.DOI
  5. [5]R. E. Alcouffe, “Diffusion synthetic acceleration for diamond-differenced discrete ordinates equations,” Nuclear Science and Engineering, vol. 64, pp. 344-355, 1977.
  6. [6]J. K. Patel, H. Park, C. R. E. de Oliveira, and D. A. Knoll, “Efficient Multiphysics coupling for fast burst reactors in slab geometry,” Journal of Computational and Theoretical Transport, vol. 43, pp. 289-313, 2014.DOI
  7. [7]M. L. Adams, and E. W. Larsen, “Fast iterative methods for discrete ordinates particle transport calculations,” Progress in Nuclear Energy, vol. 40, pp. 3-159, 2002.DOI
  8. [8]J. E. Morel, and T. A. Manteuffel, “An angular multigrid acceleration technique for SN equations with highly forward peaked scattering”, Nuclear Science and Engineering, vol. 104, no. 4, pp. 330-342, 1991.DOI
  9. [9]J. K. Patel, J. S. Warsa, and A. K. Prinja, “Accelerating the solution of SN equations with highly forward peaked scattering using the Fokker Planck approximation”, Annals of Nuclear Energy, vol. 147, 107665, 2020.
  10. [10]J. J. Kuczek, J. K. Patel, and R. Vasques, “Modified Fokker Planck acceleration for forward peaked transport problems in slab geometry,” Journal of Computational and Theoretical Transport, vol. 50, pp. 430-453, 2021.
  11. [11]K. M. Khattab, and E. W. Larsen, “Synthetic acceleration methods for linear transport problems with highly anisotropic scattering,” Nuclear Science and Engineering, vol. 107, no. 3, pp. 217-227, 1991.
  12. [12]E. N. Aristova, and V. Ya. Gol’din, “Computation of anisotropic scattering of solar radiation in atmosphere (monoenergetic case)”, Journal of Quantitative Spectroscopy and Radiative Transfer, vol. 67, pp. 139-157, 2000.DOI
  13. [13]J. K. Patel, B. D. Ganapol, T. A. Wareing, and M. M. Matuszak, “Preliminary assessment of AI-enhanced S₂ solutions in slab geometry”, American Association of Physicists in Medicine 66ᵗʰ Annual Meeting and Exhibition, Los Angeles, USA, 2024.
  14. [14]J. K. Patel, M. C. Schmidt, A. Magliari, and T. A. Wareing, “AI-enhanced low order solutions for dosimetry problems with spatial heterogeneities”, American Association of Physicists in Medicine Spring Clinical Meeting, Henderson, USA, 2025.
  15. [15]E. E. Lewis, and W. F. Miller, Computational methods of neutron transport. American Nuclear Society, La Grange Park, USA, 1993.
  16. [16]L. Lu, P. Jin, G. Peng, Z. Zhang, and G. Em. Karniadakis, “Learning nonlinear operators via DeepONets based on universal approximation theorem of operators,” Nature Machine Intelligence, vol. 3, pp. 218-229, 2021.
  17. [17]Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, “Fourier neural operator for parametric partial differential equations,” In proc. The Ninth International Conference on Learning Representations, 2021. Available: arxiv.org/pdf/2010.08895.
  18. [18]S. Sra, S. Nowozin, and S. J. Wright, Optimization for machine learning. MIT Press, Cambridge, USA, 2012.
  19. [19]I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT Press, Cambridge, USA, 2016.
  20. [20]J. Bradbury et al., JAX: composable transformations of Python+NumPy programs, http://github.com/google/jax, 2018.
  21. [21]D. P. Kingma, and J. Ba, “Adam: a method for stochastic optimization,” In Proc. International Conference on Learning Representations, San Diego, USA, 2015.
  22. [22]L. C. Henyey, and J. L. Greenstein, “Diffuse radiation in galaxy,” Journal of Astrophysics, vol. 93, pp. 70-83, 1941.

Paper details

vlnc:W_tx4zdqgd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents