betaFeedback

WorksComputer Science

MC2^2: Monte Carlo Correction for Fast Elliptic PDE Solving

Abstract

Partial differential equation (PDE) solvers underpin scientific computing, but real-world deployment is bounded by compute. Classical Monte Carlo solvers such as Walk-on-Spheres (WoS) are unbiased and geometry-agnostic but are slow. Learned solvers are fast but biased and brittle under distribution shift. We present \textbf{MC2^2}, a hybrid WoS-Neural Network (WoS-NN) PDE solver that treats a low-budget Monte Carlo solution as a structured estimator of the true field and learns a single-pass neural correction to recover a high-fidelity solution. MC2^2 matches the accuracy of solutions using over 1000×1000\times more Monte Carlo compute, outperforming all evaluated classical, denoising, and neural-operator baselines. To enable reproducible study of finite-compute PDE solving, we additionally release PDEZoo, the largest standardized elliptic PDE benchmark to date: 2M PDEs spanning five elliptic families and unlimited geometric compositions, with analytic ground truth and multi-budget Monte Carlo trajectories. Together \textbf{MC2^2} and PDEZoo (1) empirically establish that finite-sample Monte Carlo error is structured, learnable, and correctable in a single forward pass, (2) show that we can solve PDEs ∼\sim1000x faster than with just WoS, and (3) provide the evaluation infrastructure the field has so far lacked.

Introduction

Elliptic partial differential equations (PDEs) are a core computational primitive across scientific computing, geometry processing, and computer graphics, critical for tasks such as reconstruction, simulation, and physical modeling [3, 6, 15]. In practice, solvers must operate reliably across diverse forcing functions, boundary conditions, and geometries, while remaining fast enough for large-scale or latency-sensitive pipelines. Despite decades of progress, we lack approaches that simultaneously achieve generality, accuracy, and efficiency in this regime.

Classical deterministic solvers such as finite difference and finite element methods (FDM/FEM) can be highly accurate, but rely on explicit meshing and problem-specific discretization choices that limits flexibility on irregular domains [5, 4, 30]. Monte Carlo solvers, most notably Walk-on-Spheres (WoS), address many of these limitations. They are mesh-free, unbiased, and applicable to broad classes of elliptic PDEs and geometries [27, 30, 31]. However, their finite-iteration outputs suffer from high variance, requiring orders of magnitude more samples to reach acceptable accuracy [32].

Learning-based solvers, such as PINNs, neural operators, transformer models, offer fast inference but introduce bias, and can fail under distribution shifts or unseen PDE specifications [1, 21, 7, 23]. Recent hybrid approaches integrate neural networks into Monte Carlo solvers (e.g. neural caches [38]) or condition generative models on sparse numerical samples [16], but require per-problem retraining, slow iterative inference, or both.

In this work, we propose MC2^{2} (Monte Carlo error correction with Monte Carlo solvers): a new approach to hybrid PDE solving that treats the low-iteration output of a Monte Carlo method as

Preprint. a structured estimator of the true solution and learns a neural network correction that reduces error in a single forward pass. Rather than modifying a Monte Carlo solver or replacing it with a learned surrogate, MC² applies a transformer-based field-to-field model to early Walk-on-Spheres outputs, recovering solution quality that would otherwise require around 1000x more Monte Carlo iterations [10]. This preserves the generality and robustness of Monte Carlo solvers while dramatically accelerating convergence and lowering the bias of standalone neural solvers.

To support this setting and enable reproducible evaluation, we introduce PDEZoo, a benchmark offering nearly limitless diversity within the elliptic PDE regime. PDEZoo spans Laplace, Poisson, Yukawa, Biharmonic, and Helmholtz equations with explicit forcing functions, boundary conditions, and dense reference solutions. Unlike existing PDE benchmarks, PDEZoo is designed for finite-compute evaluation: it includes Monte Carlo solver trajectories at multiple compute budgets, exposing how solvers behave before asymptotic convergence and allowing methods to be compared under practical resource constraints. We further release the full procedural generator, enabling arbitrarily large training sets while preserving a fixed benchmark for standardized evaluation.

Our contributions are twofold:

  1. MC², a hybrid WoS–NN solver that corrects structured Monte Carlo error across diverse elliptic PDEs, matching the quality of using 512–2,048x more Monte Carlo compute and outperforming all evaluated baselines. In other words, it combines the generality and unbiasedness of Monte Carlo PDE solvers with learned error correction to yield a more than 1000x wall-clock speedup on PDE solving without sacrificing robustness.

  1. PDEZoo, the largest standardized benchmark to date for elliptic PDEs with explicit forcing and boundary functions, augmented with Monte Carlo solver trajectories and a scalable procedural generator.

Related Work

Monte Carlo PDE Solvers: Walk-on-Spheres (WoS) estimates solutions to elliptic PDEs via random walks guided by Green’s functions [27]. Since WoS is mesh-free and unbiased, it extends to irregular geometries and has been generalized to mixed Dirichlet–Neumann conditions [32], boundary integral formulations [33], and Monte Carlo finite element settings [19]. These properties make WoS attractive for graphics and simulation [30, 31]. However, the estimator variance decays only as O(1/K)O(1/K), meaning that the typical estimation error decays only as O(1/K)O(1/\sqrt{K}). As a result, reaching acceptable accuracy in budget-constrained settings can require prohibitively many samples.

Neural Networks for PDEs: Physics-informed neural networks (PINNs) enforce PDE residuals through specialized loss functions [1, 18, 28], while neural operators such as FNO learn mappings between function spaces [24]. More recent work has explored generative approaches using diffusion models. DiffusionPDE conditions a diffusion model on sparse observations to jointly recover coefficients and solutions while CoCoGen augments score-based sampling with PDE-residual guidance to improve physical consistency [16, 17]. Transformer-based architectures have further enabled fast inference and some degree of cross-PDE generalization. However, neural solvers face non-convex loss landscapes, spectral bias [37, 29], and distribution-shift fragility, producing biased solutions that degrade on out-of-distribution problems [23, 7].

Hybrid Monte Carlo-Neural Network Methods: Recent work combines neural networks with Monte Carlo solvers to retain unbiasedness while reducing variance. Li et al. [23] train a neural field to approximate the PDE solution, then use it as a control-variate cache during WoS sampling. While effective, this requires per-problem training and manual hyperparameter tuning. Diffusion-based approaches condition generative models on sparse numerical samples to reconstruct full solution fields [16], but suffer from slow multi-step inference and conditioning instability [20]. Our approach differs in two key ways: (i) the corrector is trained once across the entire PDE distribution rather than per-instance, and (ii) correction is a single deterministic forward pass, not iterative sampling.

Benchmarks and Scientific Progress. Despite a rapidly growing literature on neural and hybrid PDE solvers, the field lacks the unified, standardized evaluation infrastructure that has driven progress in adjacent areas of machine learning. Donoho [9] argues that “frictionless reproducibility”, fostered by shared tasks, common data, and open leaderboards, was the engine behind transformative progress in NLP and computer vision, where benchmarks like GLUE [35], SuperGLUE [36], and ImageNet [8] created a culture of head-to-head comparison on identical inputs. PDE solving has no analog. Existing benchmarks each occupy narrow regimes: PDEBench [34] and PDEArena [12] target time-dependent PDEs on fixed rectangular grids; the FNO datasets [?] ship a small number of canonical problems on the unit square; and Poseidon [14], while large-scale, is restricted to fixed grids and uses numerical solver outputs as ground truth. Crucially, none of these benchmarks support finite-compute evaluation: they provide a single converged solution per instance, with no visibility into solver behavior at intermediate sample budgets. They also lack analytic ground truth, conflating model error with discretization error from the reference solver. Furthermore, they all lack a maintained leaderboard and sufficient test sizes for the community to accurately track State-Of-The-Art results. As a result, comparisons across papers in this space are typically apples-to-oranges, and there is no shared standard against which new methods are evaluated. We designed PDEZoo to fill this gap.

Image not extracted

Figure 1. MC² pipeline. A low-budget Monte Carlo WoS estimate is corrected in a single forward pass by a learned operator, yielding an improved solution for the PDE.

Methodology: Monte Carlo Error Correction

We introduce a framework for accurate yet extremely fast elliptic PDE solving in the finite-compute regime, where classical Monte Carlo solvers are unbiased but often dominated by variance [27]. We begin by formalizing Monte Carlo PDE solvers as unbiased but stochastic estimators, then introduce a learned estimator correction framework and its neural instantiation. Importantly, our approach substantially reduces estimator variance while introducing only bounded bias, yielding a favorable bias–variance tradeoff that we analyze in Appendix § 13.

Problem Setting and PDE Families

We consider elliptic partial differential equations of the following form with Dirichlet boundary conditions:

{∇⋅(α(x)∇u(x))+ω(x)∇u(x)−σ(x)u(x)=−f(x),x∈Ωu(x)=g(x),x∈∂Ω(1)\begin{cases} \nabla\cdot\left(\alpha(x)\nabla u(x)\right) + \omega(x)\nabla u(x) - \sigma(x)u(x) = -f(x), & x \in\Omega\\ u(x) = g(x), & x \in\partial\Omega \tag*{(1)} \end{cases}

where α\alpha, ω\omega, σ\sigma are spatially varying coefficients, ff is the forcing function, gg is the boundary function, and Ω\Omega is specified via a signed distance function [23].

Monte Carlo PDE Solvers as Unbiased Stochastic Estimators

Monte Carlo solvers such as Walk-on-Spheres (WoS) express the solution u(x)u(x) as the expectation of a random process derived from Green’s functions: u(x)=S(x)+E[u(Y)]u(x) = S(x) + \mathbb{E}[u(Y)], where YY denotes the next random walk location and S(x)S(x) aggregates source terms. In practice, this expectation is approximated with KK samples: u^K(x)=1K∑k=1Ku(Yk)\hat{u}_K(x) = \frac{1}{K}\sum_{k=1}^{K} u(Y_k), which satisfies E[u^K(x)]=u(x)\mathbb{E}[\hat{u}_K(x)] = u(x) and Var⁡[u^K(x)]=O(1/K)\operatorname{Var}[\hat{u}_K(x)] = O(1/K). While unbiased, u^K\hat{u}_K is dominated by variance at small KK, severely limiting accuracy in budget-constrained regimes.

Finite-Compute Estimator Correction

Rather than modifying the Monte Carlo solver, we adopt an estimator-centric perspective and treat the finite-budget output as a noisy observation u^K=u+ϵK\hat{u}_K = u + \epsilon_K, where ϵK\epsilon_K is zero-mean, heteroskedastic, and spatially correlated noise induced by finite sampling. Thus, we seek a corrected estimator u~=Φ(u^K)\tilde{u} = \Phi(\hat{u}_K) that minimizes mean-squared error under fixed compute. Under squared loss, the optimal mapping is Φ∗(u^K)=E[u∣u^K]\Phi^*(\hat{u}_K) = \mathbb{E}[u \mid\hat{u}_K], which trades unbiasedness for variance reduction. Since Φ∗\Phi^* is intractable, we learn an approximation Φθ\Phi_\theta via:

min⁡θ E[∥Φθ(u^K)−u∥22],(2)\min_{\theta}\ \mathbb{E}\left[\left\|\Phi_\theta(\hat{u}_K)-u\right\|_2^2\right], \tag*{(2)}

using ground truth solutions as supervision. Crucially, Φθ\Phi_\theta operates purely as a post-processing correction: it does not need PDE coefficients, boundary conditions, or solver internals at inference time.

Field Discretization

For learning and evaluation, continuous solution fields are discretized on uniform grids, yielding arrays U∈RH×WU \in\mathbb{R}^{H \times W}. This discretization is used solely for numerical representation. Each training pair consists of a low-budget Monte Carlo estimator U^K\hat{U}_K and a reference solution U∗U^*. The learning problem is therefore a field-to-field regression from stochastic estimators to ground truth. We note that our method can be extended to arbitrary discretizations without retraining by evaluating the Monte Carlo estimator on a finer discretization and applying the learned correction in a sliding-window fashion.

Neural Estimator Correction

We parameterize Φθ\Phi_\theta by using a modified SUNet (Swin-UNet), a transformer-based encoder–decoder originally developed for image restoration. Its multi-scale hierarchy, shifted-window attention, and skip connections make it well-suited for modeling the long-range, spatially structured error induced by Monte Carlo sampling. At inference time, we run a Monte Carlo solver at budget KK to obtain u^K\hat{u}_K and apply the learned correction u~=Φθ(u^K)\tilde{u} = \Phi_\theta(\hat{u}_K) in a single forward pass. The correction step incurs constant-time overhead.

PDEZoo: A Large-Scale Benchmark for Elliptic PDEs

We introduce PDEZoo, a large-scale benchmark for elliptic partial differential equations (PDEs). It is intended to evaluate numerical, learning-based, and hybrid solvers in the finite-compute regime. Each instance provides (i) an explicit elliptic boundary-value problem with Dirichlet boundary conditions, (ii) a known analytic solution used as ground truth, and (iii) optional multi-budget trajectories produced by a Monte Carlo solver.

PDEZoo is the only benchmark combining exact analytic ground truth, diverse per-instance geometries, multi-budget stochastic solver data, and a scalable procedural generator (Table 1), allowing us to benchmark solver performance at various compute levels within a unified and reproducible evaluation protocol.

BenchmarkPDE ScopeGeometriesAnalytic Ground TruthIntermediate SolutionsProcedural GeneratorScale
PDEBench [34]Time-dep. PDEsFixed grids✗✗✗∼\sim10k
PDEArena [12]Time-dep. PDEsFixed grids✗✗✗∼\sim5–10k
FNO datasets [24]Canonical PDEsUnit square✗✗✗∼\sim 7k
Poseidon [14]Multi-PDEFixed grids✗✗✗1.6M
PDEZooElliptic (5 families)Unlimited (8 primitives)✓✓✓2M

Table 1. Comparison of PDE benchmarks. PDEZoo is the only benchmark combining exact analytic ground truth, diverse per-instance geometries, multi-budget stochastic solver data, and a scalable procedural generator.

Image not extracted

Figure 2. Overview of Data Distribution PDEZoo

Problem Specification

Each PDE instance is defined by a tuple P=(Ω,E,f,g,u)\mathcal{P} = (\Omega, \mathcal{E}, f, g, u), where Ω⊂Rd\Omega\subset\mathbb{R}^{d} is a bounded domain with boundary ∂Ω\partial\Omega, E\mathcal{E} denotes an elliptic PDE family, f:Ω→Rf : \Omega\to\mathbb{R} is a forcing term, g:∂Ω→Rg : \partial\Omega\to\mathbb{R} specifies Dirichlet boundary conditions, and u:Ω→Ru : \Omega\to\mathbb{R} is the analytical solution. PDEZoo includes five elliptic PDE families:

Laplace:Δu=0,Poisson:Δu=f,Yukawa:Δu−λu=f,Biharmonic:Δ2u=f,Helmholtz:Δu+k2u=f.\begin{aligned} \text{Laplace:}\quad&\Delta u = 0, \qquad\text{Poisson:}\quad\Delta u = f, \qquad\text{Yukawa:}\quad\Delta u - \lambda u = f, \\ \text{Biharmonic:}\quad&\Delta^{2}u = f, \qquad\text{Helmholtz:}\quad\Delta u + k^{2}u = f. \end{aligned}

all posed in Ω\Omega with Dirichlet boundary condition u=gu = g on ∂Ω\partial\Omega. For each family, uu is analytically specified and the corresponding right-hand side ff and boundary values gg are derived exactly from uu. Additional details on the manufactured solution families and parameter ranges are provided in Appendix § 8.§ 7.

PDEZoo Generation

Each instance is generated by the following procedure. First, we sample a PDE family (Laplace, Poisson, or Yukawa, each with probability 1/31/3) and a domain Ω\Omega, with the latter drawn uniformly from one of eight primitive shapes or a Boolean composition of two primitives. We then construct an analytic solution uu via the method of manufactured solutions: uu is defined as a random sum of nn atom functions drawn from a curated pool, and the forcing term ff and boundary condition gg are derived analytically from uu and the selected PDE family. This yields a fully specified boundary value problem with a known analytic solution.

Solution atoms. The atom pool is designed to span the spectrum of features that arise in real elliptic PDE solutions: smooth low-frequency modes (polynomials, plane waves), localized structures (Gaussian bumps, radial basis functions), singularity-like behavior (log sources, rational functions), and sharp gradients (tanh transitions, high-frequency products). Using these atoms, PDEZoo contains 2 million elliptic PDE instances across Laplace, Poisson, Yukawa, Helmholtz, and Biharmonic families.

Domains. Domains are represented implicitly through signed distance functions (SDFs), which support geometry evaluation, boundary projection, and interior/boundary sampling. PDEZoo includes eight primitive domain families (disk, square, rectangle, ellipse, annulus, triangle, hexagon, and stadium) as well as composed domains formed by Boolean operations (union, intersection, difference) on two or more primitives. Composed SDFs are conservative underestimates of the true boundary distance, preserving WoS correctness (Appendix § 8.§ 2).

Ground truth. Ground-truth labels are given by evaluating the analytic solution uu on a fixed 256×256256 \times256 grid over pixel space.

Finite Compute Monte Carlo Partial Solutions

For each PDE instance, PDEZoo includes intermediate Monte Carlo WoS solver outputs at a set of budgets B={B1<⋯<Bm}\mathcal{B} = \{B_{1} < \cdots< B_{m}\}, producing a trajectory {u^B}B∈B\{\hat{u}_{B}\}_{B \in\mathcal{B}} evaluated on the same grid as uu. These trajectories expose finite-sample estimator behavior and allow models to be trained or evaluated using intermediate solutions. Budget sets, storage format, and reproducible generation details are in Appendix A.4.

Image not extracted

Figure 3. Qualitative comparison of MC2MC^{2} against baselines on a Poisson instance. MC2MC^{2} closely matches analytic ground truth and neural operators perform poorly. DiffusionPDE, CoCoGen, and raw WoS at B=8B = 8 all retain substantial noise, and the best traditional denoiser over-smooths boundaries.

Splits, Tasks, and Evaluation Protocol

Fixed benchmark splits. PDEZoo releases fixed splits for full reproducibility and to prevent training on test set data. The train split contains 2,000,000 Laplace, Poisson, and Yukawa instances with WoS solutions at budgets Btrain=1,2,4,8,16,32B_{\mathrm{train}} = 1, 2, 4, 8, 16, 32. The out-of-distribution test split contains 5,000 instances across Laplace, Poisson, Yukawa, Helmholtz, and Biharmonic, with WoS solutions at 20 budgets from B=1B = 1 to B=131,072B = 131{,}072. Helmholtz and Biharmonic instances are reserved for testing to evaluate generalization across PDE types. Each instance stores three 256×256256 \times256 float32 arrays per budget: the WoS estimate, analytic ground truth, and domain mask.

Evaluation Settings. PDEZoo supports three evaluation settings on each PDE type:

(a) Dense-field solving: predict uu on the evaluation grid given (Ω,E,f,g)(\Omega, E, f, g)

(b) Finite-compute correction: given a low-budget estimate u^B\hat{u}_B, produce an improved estimate under fixed compute.

(c) Point-query evaluation: estimate u(x)u(x) at queried points x∈Ωx \in\Omega.

Difficulty calibration. A useful benchmark requires a controlled spread of instance difficulty. We classify instances into four tiers based on MSE at the reference budget B=32B = 32 (Table 8). See Appendix A.6 for more details.

Experiments

Training Regime

We train all methods on the same random subset of 100,000 elliptic PDE from PDEZoo’s training split. Each instance consists of a PDE specification (forcing function ff, boundary condition gg, and domain Ω\Omega), a Monte Carlo Walk-on-Spheres (WoS) solution at budget of B=8B = 8, and the corresponding analytic ground truth solution uu. We use a subset of 5,000 PDEs from the PDEZoo training split for training validation, and use it for early stopping and hyperparameter selection.

For MC2MC^{2}, we train using data from MC budgets of B∈{2n}n=25B \in\{2^{n}\}_{n=2}^{5} and their respective source terms, with an error-correction objective using our analytical ground truth as the target output. During inference, the B=8B = 8 WoS solutions (noisy estimates) and the source term are inputted into the model.

For baselines trained to learn operators, the inputs are the forcing functions ff evaluated on the same grid. For diffusion baselines, we train a separate EDM teacher per PDE family on the joint distribution (u,f,λ,mask)(u, f, \lambda, \mathrm{mask}) with the standard score-matching loss and DiffusionPDE or CoCoGen as the sampler so no MC observation enters training.

Comparisons

Walk-on-Spheres (WoS). We evaluate raw WoS solutions across a range of sampling budgets: B∈{8,16,32,64,256,1,024,4,096,16,384}B \in\{8, 16, 32, 64, 256, 1{,}024, 4{,}096, 16{,}384\} walks per grid point. These provide unbiased but variance-dominated estimates of the true solution, with accuracy improving on the order of O(1/B)O(1/\sqrt{B}).

Traditional Denoisers. We apply six classical denoising algorithms to budget B=8B = 8 WoS outputs: gaussian filtering, median filtering, bilateral filtering, total variation minimization, and non-local means. These methods assume generic noise models and do not leverage PDE-specific structure.

Neural Operator Models. We evaluate FNO [24] and PINO [25] in the operator learning setting, where the input is the forcing function ff and the target is the solution uu. PINO uses the same FNO architecture with an additional physics loss term,

L=Ldata+λLphysics,λ=0.1,\mathcal{L} = \mathcal{L}_{\mathrm{data}} + \lambda\mathcal{L}_{\mathrm{physics}}, \qquad\lambda= 0.1,

where Ldata\mathcal{L}_{\mathrm{data}} is the MSE to ground truth and Lphysics\mathcal{L}_{\mathrm{physics}} penalizes Laplacian inconsistency.

Diffusion Models. We evaluate DiffusionPDE and CoCoGen, two SOTA PDE solvers built on a per-PDE-family EDM. Since these models must be trained on a particular PDE family and are not generalizable, we report results only for PDE families in our train set for these models: Laplace, Poisson, and Yukawa.

Evaluation Results

We evaluate on the 5,000-instance PDEZoo test split and present our quantitative results across all methods in Table 2. Our MC² corrector achieves 44.10dB PSNR from budget B=8B = 8 input, which is an improvement of 31.6 dB over the raw B=8B = 8 WoS estimate (25.54 dB) and surpassing WoS at B=16,384B = 16{,}384 (2048× more compute). MC² outperforms all other methods such as FNO [24], PINO [25], and CoCoGen [17] across all metrics. We also show strong scaling at every metric

MethodPSNR (dB) ↑SNR (dB) ↑MSE ↓LPIPS ↓
Walk-on-Spheres (Monte Carlo)
WoS (B=8B = 8)15.26 ± 7.439.773 ± 7.527(8.02±8.34)×10−2(8.02 \pm8.34) \times10^{-2}0.7275 ± 0.4198
WoS (B=16384B = 16384)42.14 ± 13.8236.644 ± 13.843(3.13±12.43)×10−3(3.13 \pm12.43) \times10^{-3}0.1964 ± 0.3088
WoS (B=32768B = 32768)44.98 ± 13.9239.486 ± 13.935(1.88±8.77)×10−3(1.88 \pm8.77) \times10^{-3}0.1592 ± 0.2731
WoS (B=65536B = 65536)47.80 ± 14.0042.307 ± 14.009(1.11±6.15)×10−3(1.11 \pm6.15) \times10^{-3}0.1263 ± 0.2381
Traditional Denoisers (applied to B=8B = 8 WoS)
Gaussian18.85 ± 8.3113.361 ± 8.253(5.06±6.96)×10−2(5.06 \pm6.96) \times10^{-2}0.4032 ± 0.2520
Gaussian (Heavy)17.66 ± 6.4812.164 ± 6.307(4.06±5.04)×10−2(4.06 \pm5.04) \times10^{-2}0.2832 ± 0.2062
Total Variation24.63 ± 10.4119.141 ± 10.419(2.71±4.86)×10−2(2.71 \pm4.86) \times10^{-2}0.2613 ± 0.2123
Non-Local Means26.63 ± 10.3621.134 ± 10.444(1.93±3.94)×10−2(1.93 \pm3.94) \times10^{-2}0.3212 ± 0.3014
Neural Network Methods
FNO7.74 ± 5.472.251 ± 4.846(2.58±1.57)×10−1(2.58 \pm1.57) \times10^{-1}0.3790 ± 0.2277
PINO8.24 ± 6.132.745 ± 5.878(2.49±1.57)×10−1(2.49 \pm1.57) \times10^{-1}0.3753 ± 0.2236
CoCoGen8.25 ± 2.922.674 ± 3.631(1.78±0.91)×10−1(1.78 \pm0.91) \times10^{-1}0.5597 ± 0.2522
DiffusionPDE8.25 ± 2.922.674 ± 3.632(1.78±0.91)×10−1(1.78 \pm0.91) \times10^{-1}0.5600 ± 0.2525
MC² (Ours)44.90 ± 12.4939.410 ± 12.532(1.94±11.38)×10−3(1.94 \pm11.38) \times10^{-3}0.0164 ± 0.0681

Table 2. MC² matches the accuracy of orders-of-magnitude more Monte Carlo compute and outperforms all other neural network methods on PDEZoo. Evaluation are on the 5,000-instance PDEZoo test set, with metrics averaged across all five elliptic PDE families (Biharmonic, Helmholtz, Laplace, Poisson, and Yukawa). Arrows indicate best metric direction. Bold marks the best result per column among learned and post-processing methods.

Generalization to Out-of-Distribution (OOD) PDE Families: Additionally, we show that MC² generalizes well to OOD PDE families, as seen in Table 3, which is a huge advantage over previous SOTA diffusion-based methods who are unable to generalize outside single PDE families.

MethodLaplacePoissonYukawaAverageBiharmonicHelmholtzAverage
In-distributionOut-of-distribution
Walk-on-Spheres (Monte Carlo)
WoS (B=8B = 8)25.22±3.3325.22 \pm3.3311.20±5.2511.20 \pm5.2513.89±5.8813.89 \pm5.8816.82±7.8416.82 \pm7.8415.01±6.3815.01 \pm6.3810.93±5.0110.93 \pm5.0112.96±6.0912.96 \pm6.09
WoS (B=8192B = 8192)55.29±3.4255.29 \pm3.4232.94±11.7332.94 \pm11.7338.83±10.0838.83 \pm10.0842.41±13.1842.41 \pm13.1837.30±13.7637.30 \pm13.7632.05±11.8832.05 \pm11.8834.67±13.1234.67 \pm13.12
WoS (B=16384B = 16384)58.30±3.4258.30 \pm3.4235.86±11.8535.86 \pm11.8541.81±10.1141.81 \pm10.1145.38±13.2545.38 \pm13.2539.77±13.9139.77 \pm13.9134.89±12.0834.89 \pm12.0837.32±13.2537.32 \pm13.25
WoS (B=32768B = 32768)61.31±3.4261.31 \pm3.4238.81±11.9238.81 \pm11.9244.80±10.1244.80 \pm10.1248.37±13.2948.37 \pm13.2942.15±13.9842.15 \pm13.9837.77±12.2237.77 \pm12.2239.95±13.3139.95 \pm13.31
Traditional Denoisers (applied to B=8B = 8 WoS)
Gaussian29.67±2.5629.67 \pm2.5614.64±6.5714.64 \pm6.5717.57±6.0917.57 \pm6.0920.68±8.4620.68 \pm8.4618.11±7.5918.11 \pm7.5914.21±6.4014.21 \pm6.4016.15±7.2916.15 \pm7.29
Gaussian (Heavy)25.98±3.3025.98 \pm3.3014.64±4.9514.64 \pm4.9516.55±4.8716.55 \pm4.8719.10±6.6719.10 \pm6.6716.73±5.8216.73 \pm5.8214.32±4.9814.32 \pm4.9815.52±5.5515.52 \pm5.55
Total Variation37.81±2.6337.81 \pm2.6319.34±8.6019.34 \pm8.6023.34±7.7723.34 \pm7.7726.89±10.5126.89 \pm10.5123.81±9.6223.81 \pm9.6218.80±8.2918.80 \pm8.2921.29±9.3221.29 \pm9.32
Non-Local Means39.16±3.0539.16 \pm3.0521.56±8.8521.56 \pm8.8525.54±7.7125.54 \pm7.7128.81±10.3128.81 \pm10.3125.91±9.9025.91 \pm9.9020.88±8.4820.88 \pm8.4823.39±9.5623.39 \pm9.56
Neural Network Methods
FNO16.52±5.9416.52 \pm5.945.47±1.845.47 \pm1.845.68±1.935.68 \pm1.939.27±6.419.27 \pm6.415.43±2.365.43 \pm2.365.52±1.995.52 \pm1.995.47±2.185.47 \pm2.18
PINO18.46±6.1518.46 \pm6.155.71±1.955.71 \pm1.955.65±1.895.65 \pm1.8910.00±7.2010.00 \pm7.205.54±2.325.54 \pm2.325.70±2.195.70 \pm2.195.62±2.265.62 \pm2.26
CoCoGen7.15±3.687.15 \pm3.688.78±2.218.78 \pm2.218.85±2.308.85 \pm2.308.25±2.928.25 \pm2.92–––
DiffusionPDE7.15±3.687.15 \pm3.688.78±2.218.78 \pm2.218.84±2.308.84 \pm2.308.25±2.938.25 \pm2.93–––
MC2^2 (Ours)59.94±3.8559.94 \pm3.8542.17±9.5442.17 \pm9.5446.40±8.2146.40 \pm8.2149.55±10.7549.55 \pm10.7535.96±13.2535.96 \pm13.2540.05±9.4240.05 \pm9.4238.01±11.6738.01 \pm11.67

Table 3. Per-family PSNR (dB, ↑\uparrow) on the PDEZoo test set. MC2^2 achieves the highest PSNR on every PDE family compared to other Neural Network methods, including the held-out Biharmonic and Helmholtz families absent from the training distribution. CoCoGen and DiffusionPDE are trained per-family on in-distribution PDEs only and cannot be evaluated on held-out families. MC2^2 uses a single B=8B = 8 WoS input. Bold marks the best result per column among learned and post-processing methods. Full per-family breakdown on SNR, MSE and LPIPS can be found in Appendix § 12.§ 1.

Image not extracted

Figure 4. Scaling Study of MC2^2 (a) MC2^2 performance scales smoothly with training-set size at fixed input budget B=8B = 8, with no sign of saturation. (b) We evaluate performance of applying MC2^2 error corrector to different WoS budgets. At every WoS budget, MC2^2 achieves substantially lower MSE than input WoS budget, matching the accuracy of ∼1,000×\sim1{,}000\times more MC compute. (c) On the accuracy–compute Pareto frontier, MC2^2 dominates all evaluated baselines across the entire wall-clock range.

MC2 Ablations

Multi-budget vs. single-budget training. Our MC2^2 model is trained jointly on noisy Walk-on-Spheres solutions at budgets B∈{1,2,4,8,16,32}B \in\{1, 2, 4, 8, 16, 32\}. We compare this model against single-budget baselines, which are trained using only one input budget. When evaluating on our test set where the input is a single noisy Walk-on-Spheres solution at B=8B = 8, the multi-budget model improves performance by roughly 0.7–0.8 dB.

Source-term conditioning. To quantify the importance of conditioning on the PDE source term ff, we compare models trained without ff against the full MC2^2 model at the same budget. In particular, removing ff in reduces performance by approximately 6.8 dB at B=4B = 4.

Loss precision. We compare two models trained at B=8B = 8, one of which accumulates the loss in FP32, and the other which uses BF16. The FP32-loss model reaches 57.7 dB PSNR, compared to 54.8 dB for the BF16-loss model.

Data scaling. We train five MC2^2 models using 4%, 8%, 16%, 32%, and 64% of the training set. All runs use the same hyperparameters, so differences in performance primarily reflect the amount of available training data. These runs trace a data-scaling curve for MC2^2 at a fixed Monte Carlo input budget.

Discussion

Finite-sample Monte Carlo error is structured, learnable, and correctable. The central empirical claim of this work is that the residual εK=u^K−u\varepsilon_K = \hat{u}_K - u produced by a low-budget WoS solver is not well-modeled as spatially independent noise, but carries structure that is consistent across PDE instances and recoverable by a learned operator. Three observations support this, each ruling out a distinct alternative hypothesis:

  • Against an unstructured-noise model. Generic denoisers that assume locally-stationary noise (Gaussian, total variation, non-local means) plateau below 27 dB on B=8B = 8 inputs, while MC2MC^2 reaches 44.10 dB on the same inputs (Table 2). Since the baselines are non-parametric, the 17 dB gap is a measured lower bound on the information content of εK\varepsilon_K that pointwise noise models cannot exploit.

  • Against a pure solution-prior explanation. FNO trained on the same data with direct access to (f,g,Ω)(f, g, \Omega) achieves 30.34 dB, which is 13.76 dB below MC2MC^2 despite having strictly more direct information about the target. The corrector therefore exploits joint statistics of (u^K,u)(\hat{u}_K, u) that are not recoverable from the PDE specification alone.

  • Against a family-specific structure. MC2MC^2 generalizes zero-shot to held-out Biharmonic and Helmholtz families (34.88 and 35.87 dB), within 4–5 dB of its in-distribution Poisson performance. Because these operators are qualitatively different (fourth-order and indefinite), the transfer is consistent with the corrector having learned features of the WoS estimator’s finite-sample behavior, such as boundary-induced correlations, geometry-dependent variance scaling, rather than family-specific solution priors.

We emphasize the scope of this claim. Our evidence establishes that finite-sample WoS error on 2D elliptic problems contains learnable structure sufficient to recover ∼30\sim30 dB from a single B=8B = 8 estimate; we do not claim the structure admits a closed-form description, that the learned corrector is Bayes-optimal, or that comparable gains transfer to arbitrary stochastic solvers. The bias–variance analysis in Appendix § 13 formalizes when a Lipschitz corrector strictly reduces MSE relative to the underlying unbiased estimator, but the empirical magnitude is a property of the WoS–elliptic-PDE pair we study.

Beyond Walk-on-Spheres: A General Recipe for Unbiased Estimators. Although we instantiate Monte Carlo error correction with WoS, nothing in the framework is specific to PDEs or to walk-based estimators. The key requirements are simply (i) an unbiased stochastic estimator u^K\hat{u}_K whose finite-budget error has learnable structure, such as spatially or temporally correlated bias-free noise induced by geometry, sampling distribution, or problem parameters, and (ii) access to high-fidelity reference outputs or ground truths for supervision. These conditions are met by a wide range of estimators across scientific computing and machine learning: path-traced rendering, Markov chain Monte Carlo posterior sampling, particle filters, neutron transport simulations, and quantum Monte Carlo. In each case, finite-sample error exhibits the same kind of structured, distribution-specific patterns that a learned corrector can exploit. We view MC2MC^2 as the first instance of a broader paradigm: train once across the problem distribution, then convert any unbiased sampler into a fast, single-pass estimator while retaining bounded unbiasedness from the underlying unbiased solver. Extending the framework to these other domains is an immediate direction for future work.

Conclusion

We make two contributions to elliptic PDE solving in the finite-compute regime. MC2MC^2 establishes that finite-sample Monte Carlo error is structured and correctable in a single forward pass, achieving the accuracy of over 1000x more sampling compute and outperforming all evaluated baselines, including on out-of-distribution PDE families. PDEZoo provides the infrastructure the field has so far lacked: 2M elliptic PDEs with analytic ground truth and multi-budget Monte Carlo trajectories. A main limitation is that we can only solve the class of PDEs that Walk-on-Spheres can solve, which are broadly elliptic problems on domains where the Green’s-function machinery applies, and extending the framework to hyperbolic, parabolic, or strongly nonlinear regimes will require pairing it with a different unbiased estimator. Within that scope, however, MC2MC^2 closes most of the gap to high-budget Monte Carlo at a fraction of the cost, and we see the same recipe as a template for the broader family of unbiased Monte Carlo solvers.

[37] Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.

[38] Rui Zhang, Qi Meng, Rongchan Zhu, Yue Wang, Wenlei Shi, Shihua Zhang, Zhi-Ming Ma, and Tie-Yan Liu. Monte carlo neural pde solver for learning pdes via probabilistic representation, 2023.

PDEZoo: Detailed Specifications

Solution Atom Types

Each solution u(x,y)u(x,y) is constructed as a sum of nn randomly chosen atoms, where nn is sampled uniformly from [2,6][2,6]. For Poisson and Yukawa instances, atoms are drawn from the general pool (20 types); for Laplace instances, atoms are drawn from the harmonic pool (12 unique types). For Poisson and Yukawa instances, atoms are drawn from a pool of 20 such types. For Laplace instances, atoms are restricted to a pool of 12 analytically harmonic types — real and imaginary parts of holomorphic functions such as znz^n, ekze^{kz}, log⁡(z−z0)\log(z-z_0), and 1/(z−z0)n1/(z-z_0)^n — guaranteeing ∇2h=0\nabla^2 h = 0 by construction. High-frequency and sharp-featured atoms are deliberately over-represented to produce instances where Monte Carlo variance is large and correction is non-trivial, yielding a calibrated difficulty distribution (Appendix § 8.§ 1). Combinatorially, choosing 2–6 atoms from these pools yields over 60,000 distinct type tuples for Poisson/Yukawa alone; each tuple is further parameterized by continuous amplitudes, frequencies, and source locations, making the effective solution space inexhaustible.

General atoms (Poisson/Yukawa). Table 4 lists all 20 general atom types. Five of these are designated as hard atoms (marked with †), producing high-frequency oscillations, narrow features, or sharp transitions that increase Monte Carlo variance. Hard atoms are over-represented in the selection pool via the hard_atom_extra parameter (set to 4), which appends 4 extra copies of each hard atom, raising their draw probability from 25% to 63%.

AtomFormKey Parameters
polyPolynomial (degree ≤3\le3)Coefficients in [−1,1][-1,1]
trigasin⁡(kπx)a\sin(k\pi x)k∈[1,6]k\in[1,6]
expaexp⁡(bx)a\exp(bx)b∈[−2,2]b\in[-2,2]
logalog⁡(x2+0.01)a\log(x^2+0.01)—
hyperasinh⁡/cosh⁡/tanh⁡(x)a\sinh/\cosh/\tanh(x)—
specialaerf⁡(x)a\operatorname{erf}(x) or ax2+ϵa\sqrt{x^2+\epsilon}—
harmonic_polararncos⁡(nθ)ar^n\cos(n\theta)n∈[1,6]n\in[1,6]
plane_waveacos⁡(kn^⋅x)a\cos(k\hat{n}\cdot x)k∈[1,8]k\in[1,8]
yukawa_i0aI0(μr)aI_0(\mu r)μ\mu from λ∈[0.5,6]\lambda\in[0.5,6]
log_sourcealog⁡∣x−x0∣a\log\lvert x-x_0\rvertρ∈[1.8,2.5]\rho\in[1.8,2.5]
gaussian_bumpaexp⁡(−w∥x−x0∥2)a\exp(-w\lVert x-x_0\rVert^2)w∈[0.8,2]w\in[0.8,2]
rationala/(1+bx2+cy2)a/(1+bx^2+cy^2)b,c∈[0.5,3]b,c\in[0.5,3]
productaf1(k1x)f2(k2y)af_1(k_1x)f_2(k_2y)fi∈{sin⁡,cos⁡,tanh⁡}f_i\in\{\sin,\cos,\tanh\}
multi_rbfSum of 2–4 Gaussiansw∈[2,8]w\in[2,8]
high_freq_trig†asin⁡(k1πx)cos⁡(k2πy)a\sin(k_1\pi x)\cos(k_2\pi y)k1,k2∈[2,8]k_1,k_2\in[2,8]
log_sumexpalog⁡addexp⁡(x)a\log\operatorname{addexp}(x)b,c∈[1,4]b,c\in[1,4]
sech_bump†a/(cosh⁡2b(x−x0)cosh⁡2c(y−y0))a/(\cosh^2 b(x-x_0)\cosh^2 c(y-y_0))b,c∈[2,6]b,c\in[2,6]
very_high_freq_trig†asin⁡(k1πx)cos⁡(k2πy)a\sin(k_1\pi x)\cos(k_2\pi y)k1,k2∈[6,15]k_1,k_2\in[6,15]
narrow_bump†aexp⁡(−w∥x−x0∥2)a\exp(-w\lVert x-x_0\rVert^2)w∈[8,25]w\in[8,25]
sharp_transition†atanh⁡(k(n^⋅x−c))a\tanh(k(\hat{n}\cdot x-c))k∈[5,15]k\in[5,15]

Table 4. General atom types for Poisson and Yukawa instances. † = hard atom.

Harmonic atoms (Laplace). Table 5 lists the 12 unique harmonic atom types. Every harmonic atom is the real or imaginary part of a holomorphic function f(z)f(z) where z=x+iyz=x+iy, guaranteeing ∇2h=0\nabla^2 h=0 analytically by the Cauchy–Riemann equations. Two types (marked †) constitute the hard harmonic atoms, producing solutions with high spatial frequency or signal concentrated near the boundary.

AtomFormComplex OriginWt.
h_polararncos⁡/sin⁡(nθ)ar^{n}\cos/ \sin(n\theta)Re⁡/Im⁡(zn)\operatorname{Re}/\operatorname{Im}(z^{n}), n∈[1,6]n \in[1,6]2
h_exp_trigaekxcos⁡/sin⁡(ky)ae^{kx}\cos/ \sin(ky)Re⁡/Im⁡(ekz)\operatorname{Re}/\operatorname{Im}(e^{kz}), k∈[−3,3]k \in[-3,3]2
h_trig_hypasin⁡(kx)sinh⁡(ky)a\sin(kx)\sinh(ky) etc.Separable, k∈[0.5,4]k \in[0.5,4]2
h_log_sourcealog⁡∣x−x0∣a\log\lvert x-x_{0}\rvertRe⁡(log⁡(z−z0))\operatorname{Re}(\log(z-z_{0})), ρ≥1.8\rho\ge1.82
h_linearax+byax+byRe⁡(αz)\operatorname{Re}(\alpha z)1
h_bilinearaxyaxyIm⁡(z2)/2\operatorname{Im}(z^{2})/21
h_arctanaatan2⁡(y−y0,x−x0)a\operatorname{atan2}(y-y_{0},x-x_{0})Im⁡(log⁡(z−z0))\operatorname{Im}(\log(z-z_{0}))1
h_inversiona(x−x0)/∣x−x0∣2a(x-x_{0})/\lvert x-x_{0}\rvert^{2}Re⁡(1/(z−z‾0))\operatorname{Re}(1/(z-\overline{z}_{0}))1
h_dipolea(Δx2−Δy2)/∣Δ∣4a(\Delta x^{2}-\Delta y^{2})/\lvert\Delta\rvert^{4}Re⁡(1/(z−z0)2)\operatorname{Re}(1/(z-z_{0})^{2})1
h_quadratica(u2−v2)a(u^{2}-v^{2}) or 2uv2uvRe⁡/Im⁡(z2)\operatorname{Re}/\operatorname{Im}(z^{2}) rotated2
h_high_freq_exp_trig†aekxcos⁡/sin⁡(ky)ae^{kx}\cos/ \sin(ky)Re⁡/Im⁡(ekz)\operatorname{Re}/\operatorname{Im}(e^{kz}), k∈[4,10]k \in[4,10]2
h_high_n_polar†arncos⁡/sin⁡(nθ)ar^{n}\cos/ \sin(n\theta)Re⁡/Im⁡(zn)\operatorname{Re}/\operatorname{Im}(z^{n}), n∈[6,12]n \in[6,12]2

Table 5.

Table 5: Harmonic atom types for Laplace instances. † = hard atom. Weight indicates the number of entries in the base selection pool.

Domain Geometry

PDEZoo includes 8 primitive domain families and composed domains. All domains are centered approximately at the origin; the evaluation grid spans [−1,1]2[-1,1]^{2}. Each primitive has probability 1/111/11 of being selected; composed domains have combined probability 3/113/11.

Table 6: Primitive domain types with parameter ranges and SDF properties.

DomainKey ParametersSDFExact?
DiskR=1.0R = 1.0R−∣x∣R-\lvert x\rvertYes
Squarehalf =1.0= 1.0min⁡(half−∣xi∣)\min(\mathrm{half}-\lvert x_{i}\rvert)Yes
Rectanglea,b∈[0.6,1.2]a,b \in[0.6,1.2]min⁡(a−∣x∣,b−∣y∣)\min(a-\lvert x\rvert,b-\lvert y\rvert)Yes
Ellipserx,ry∈[0.6,1.2]r_{x},r_{y} \in[0.6,1.2](1−∣x/r∣)min⁡(rx,ry)(1-\lvert x/r\rvert)\min(r_{x},r_{y})Approx.
Annulusrin∈[0.2,0.6],rout∈[0.8,1.4]r_{\mathrm{in}} \in[0.2,0.6], r_{\mathrm{out}} \in[0.8,1.4]min⁡(rout−r,r−rin)\min(r_{\mathrm{out}}-r,r-r_{\mathrm{in}})Yes
TriangleEquilateral, R∈[0.6,1.0]R \in[0.6,1.0]min⁡(signed edge dists)\min(\text{signed edge dists})Yes
HexagonRegular, R∈[0.6,1.0]R \in[0.6,1.0]min⁡(signed edge dists)\min(\text{signed edge dists})Yes
Stadiumhalf_len∈[0.2,0.6],rad∈[0.25,0.5]\mathrm{half\_len} \in[0.2,0.6], \mathrm{rad} \in[0.25,0.5]rad−dist to segment\mathrm{rad}-\text{dist to segment}Yes

Table 6.

Composed domains. Composed domains combine two randomly selected primitives (from disk, rectangle, ellipse, or triangle) via a Boolean operation: union (3/73/7), intersection (2/72/7), or difference (2/72/7). Each component has independent random parameters, rotation, and offset (separation ∈[0.15,0.55]\in[0.15,0.55]). The composed SDF is computed as max⁡(sdfA,sdfB)\max(\mathrm{sdf}_{A},\mathrm{sdf}_{B}) for union, min⁡(sdfA,sdfB)\min(\mathrm{sdf}_{A},\mathrm{sdf}_{B}) for intersection, and min⁡(sdfA,−sdfB)\min(\mathrm{sdf}_{A},-\mathrm{sdf}_{B}) for difference. These are provably conservative (i.e., sdf(x)≤dist⁡(x,∂Ω)\mathrm{sdf}(x) \le\operatorname{dist}(x,\partial\Omega)), ensuring every WoS ball lies entirely within Ω\Omega.

Quality Filters

Each generated instance must pass eight quality filters before inclusion (up to 40 retries per case):

Table 7: Quality filters applied during case generation.

FilterCondition
Finite test pointu(0.3,0.4)u(0.3,0.4) is finite
Finite forcingf(0.3,0.4)f(0.3,0.4) is finite (Poisson/Yukawa only)
Domain interior≥10\ge10 interior points in 128 random samples
Composed thicknessMedian SDF >0.03> 0.03 over interior points
Solution amplitude10−3<std⁡(u)<20010^{-3} < \operatorname{std}(u) < 200
Boundary valuesmax⁡∣u(∂Ω)∣<500\max\lvert u(\partial\Omega)\rvert< 500 (composed only)
Forcing amplitude10−4<std⁡(f)<50010^{-4} < \operatorname{std}(f) < 500 (Poisson/Yukawa only)
Finite ground truthAll GT values inside domain are finite at 256×256256 \times256

Table 7.

The WoS solver uses boundary termination threshold ε=10−4\varepsilon= 10^{-4}, introducing bias bounded by ∣bias∣≲10−3∥g∥∞\lvert\mathrm{bias}\rvert\lesssim10^{-3}\lVert g\rVert_{\infty} [2] — negligible compared to MC variance at all practical budgets. For each instance, PDEZoo also provides an executable description of uu (and hence f,gf, g) and geometry accessors (SDF evaluator, boundary projection, samplers).

Data Format

Each instance is stored as a JSONL record containing: case ID, PDE kind (laplace, poisson, or yukawa), screening parameter λ\lambda (nonzero only for Yukawa), a PyTorch expression string for the analytic solution, domain name and parameters, number of atom terms, and hardness metadata (std⁡(u),std⁡(f)\operatorname{std}(u), \operatorname{std}(f)).

Each packaged .npz file contains three (256×256)(256 \times256) float32 arrays:

  • noisy: MC solution at the given budget (0.0 outside domain)

  • clean: Ground truth analytic solution (0.0 outside domain)

  • mask: Domain indicator (1.0 inside, 0.0 outside)

Convergence Verification

We verify empirically that the WoS estimator achieves the theoretical O(1/N)O(1/N) MSE convergence rate by fitting log–log MSE slopes over budgets 1–32 across 501 test cases (167 per PDE type). Table 9 reports the results: the median slope is exactly −1.000-1.000 for all three families, and 99.6% of cases fall within [−1.05,−0.95][-1.05, -0.95]. This confirms that the manufactured solutions, derived forcing functions, and WoS implementation are mutually consistent. Convergence verification results are presented in Table 9 in the main text. The MSE of the WoS estimator satisfies MSE⁡(N)=Var⁡(single walk)/N=O(1/N)\operatorname{MSE}(N) = \operatorname{Var}(\text{single walk})/N = O(1/N). On a log–log plot, this yields slope −1.0-1.0 regardless of PDE type, domain geometry, solution difficulty, or Yukawa screening strength λ\lambda; difficulty affects only the constant (variance per walk), not the rate. Very Hard cases are almost exclusively Laplace, because the Yukawa screening weight ψ=1/I0(μR)\psi= 1/I_0(\mu R) exponentially damps estimator variance.

Table 9: Empirical WoS convergence slope verification on 501 test cases. The theoretical rate is slope =−1.0=-1.0.

PDE TypeMedian SlopeMinMax% in [−1.05,−0.95][-1.05,-0.95]
Laplace−1.000-1.000−1.015-1.015−0.981-0.981100.0%100.0\%
Poisson−1.000-1.000−1.045-1.045−0.918-0.91899.4%99.4\%
Yukawa−1.000-1.000−1.043-1.043−0.944-0.94499.4%99.4\%
All−1.000-1.000−1.045-1.045−0.918-0.91899.6%99.6\%

Table 9.

Difficulty Distribution

Difficulty tier definitions and the observed distribution are presented in Table 8 in the main text. The difficulty distribution was calibrated via a 12-configuration hyperparameter sweep varying hard-atom weighting, term count ranges, and amplitude filter caps. The selected configuration (hard_atom_extra = 4) shifts the distribution from a baseline of (34/45/17/4)%(34/45/17/4)\% to the target (20/41/30/8)%(20/41/30/8)\%, ensuring sufficient representation of challenging instances.

TierMSE @ B=32B = 32Typical CharacteristicsProportion
Easy<10−2< 10^{-2}Smooth, low-frequency, simple domains∼20%\sim20\%
Medium[10−2,1)[10^{-2}, 1)Moderate frequency, diverse domains∼41%\sim41\%
Hard[1,102)[1, 10^{2})High-freq. oscillations, strong sources∼30%\sim30\%
Very Hard≥102\ge10^{2}Very high-freq. patterns, sharp features∼8%\sim8\%

Table 8. Difficulty tier definitions and observed distribution (B=32B = 32).

PDEZoo

Each PDE instance is defined by a tuple P=(Ω,E,f,g,u)\mathcal{P} = (\Omega, \mathcal{E}, f, g, u) where Ω⊂Rd\Omega\subset\mathbb{R}^{d} is a bounded domain with boundary ∂Ω\partial\Omega, E\mathcal{E} denotes an elliptic PDE family, f:Ω→Rf : \Omega\to\mathbb{R} is a forcing term, g:∂Ω→Rg : \partial\Omega\to\mathbb{R} specifies Dirichlet boundary condition, and u:Ω‾→Ru : \overline{\Omega} \to\mathbb{R} is the analytical solution. PDEZoo includes five elliptic PDE families:

Laplace. The solution satisfies Δu=0\Delta u = 0 in Ω\Omega, with Dirichlet boundary condition u=gu = g on ∂Ω\partial\Omega. In this setting, uu is constructed from analytically harmonic atoms.

Poisson. The solution satisfies Δu=f\Delta u = f in Ω\Omega, with Dirichlet boundary values inherited from the manufactured solution. Here, uu is sampled from a rich family of smooth non-harmonic expressions, and the source term is obtained as f=Δuf = \Delta u.

Yukawa. The solution satisfies Δu−λu=f\Delta u-\lambda u=f in Ω\Omega, with screening parameter λ>0\lambda>0. The source is defined from the manufactured solution by f=Δu−λuf=\Delta u-\lambda u. In our dataset, λ\lambda is sampled uniformly from [0.5,50.0][0.5,50.0].

Biharmonic. The solution satisfies Δ2u=f\Delta^{2}u=f in Ω\Omega, with Dirichlet boundary condition u=gu=g on ∂Ω\partial\Omega. Here, uu is sampled from a family of smooth manufactured expressions, and the source term is obtained as f=Δ2uf=\Delta^{2}u, where Δ2=Δ(Δ⋅)\Delta^{2}=\Delta(\Delta\cdot) denotes the biharmonic operator.

Helmholtz. The solution satisfies Δu+k2u=f\Delta u+k^{2}u=f in Ω\Omega, with wavenumber k>0k>0 and Dirichlet boundary values inherited from the manufactured solution. The source is defined as f=Δu+k2uf=\Delta u+k^{2}u. In our dataset, kk is sampled uniformly from [0.5,10.0][0.5,10.0].

Monte Carlo Walk-On-Spheres Solver

WoS Solver Parameters

The Walk-on-Spheres solver uses the following parameters:

Table 10: WoS solver parameters.

ParameterValueDescription
Resolution256×256256\times256Uniform grid over [−1,1]2[-1,1]^{2}
ε\varepsilon (shell)10−410^{-4}Boundary termination threshold
max_steps (train)128128Maximum walk steps per sample
max_steps (eval)256256Higher for convergence verification
walk_batch256256Walks processed per GPU batch

Table 10.

ε\varepsilon-shell bias. The WoS walk terminates when the signed distance to ∂Ω\partial\Omega falls below ε=10−4\varepsilon=10^{-4}, introducing a bias bounded by ∣E[u^(x)]−u(x)∣≤Cε∣log⁡ε∣⋅∥g∥∞\lvert\mathbb{E}[\hat{u}(x)]-u(x)\rvert\le C\varepsilon\lvert\log\varepsilon\rvert\cdot\lVert g\rVert_{\infty} for 2D Laplace with Hölder-continuous boundary data [2]. At ε=10−4\varepsilon=10^{-4} this gives ∣bias∣≲10−3∥g∥∞\lvert\mathrm{bias}\rvert\lesssim10^{-3}\lVert g\rVert_{\infty}, negligible compared to the MC standard deviation σ/N\sigma/\sqrt{N} at all practical budgets.

MC2^{2} Architecture

Our MC2^{2} corrector is instantiated as a Swin-UNet (SUNet) architecture [10]. The model processes 256×256256\times256 inputs with patch size 4, embedding dimension 96, and four encoder stages with depths [8,8,8,8][8,8,8,8] and [8,8,8,8][8,8,8,8] attention heads. The decoder mirrors the encoder with skip connections. We use a window size of 8 for shifted-window self-attention, MLP ratio 4, and stochastic depth with drop path rate 0.1. The final layer uses dual upsampling to reconstruct the full-resolution output. We then train for 200 epochs using the AdamW optimizer with initial learning rate 2×10−42\times10^{-4} (decaying to 10−610^{-6}), batch size 48, and bf16 mixed precision. Training is distributed across 4 GPUs using PyTorch’s DDP. The loss function is mean squared error (MSE) between predicted and ground truth solutions, computed only on valid interior regions as defined by the domain mask. We select the checkpoint with the best validation PSNR.

Baseline Models

Denoisers

For traditional denoisers, we utilize Gaussian filtering (σ=1.0\sigma= 1.0), heavy Gaussian filtering (σ=2.0\sigma= 2.0), Median filtering (disk radius 2), Bilateral filtering (σcolor=0.1\sigma_{\mathrm{color}} = 0.1, σspatial=5\sigma_{\mathrm{spatial}} = 5), Total Variation minimization (weight 0.1), and Non-Local Means (patch size 5, search distance 6, h=0.1h = 0.1).

FNO

The architecture uses 4 Fourier layers with 16 modes and width 64. We train for 200 epochs using Adam with learning rate 10−310^{-3}, batch size 16 per GPU across 4 GPUs, and StepLR scheduling (step size 100, γ=0.5\gamma= 0.5).

Extended Results

Full Per-Family Results

For completeness, Table 11 reports per-family results across all four metrics (PSNR, SNR, MSE, LPIPS) for every method evaluated in the main paper, including the full Walk-on-Spheres compute curve from B=8B = 8 to B=131,072B = 131{,}072. The aggregate trends mirror the main-paper findings: MC2^2 from a single B=8B = 8 WoS input matches WoS quality at B∼16,384B \sim16{,}384–32,76832{,}768 on in-distribution families and at B∼8,192B \sim8{,}192–16,38416{,}384 on out-of-distribution families, corresponding to a 1,0241{,}024–4,096×4{,}096\times reduction in Monte Carlo compute. The per-family breakdown also reveals where each baseline method’s weaknesses concentrate. PINO’s physics regularization actively harms performance on every family, producing solutions worse than the raw B=8B = 8 input. Diffusion-based methods (CoCoGen, DiffusionPDE) achieve reasonable Laplace performance but degrade sharply on Poisson and Yukawa, where forcing-driven structure is harder to capture from generative priors alone. FNO is the strongest baseline overall but trails MC2^2 by 13–16 dB on every family. Among traditional denoisers, Non-Local Means and Total Variation provide the largest gains over raw WoS, but plateau well below the neural methods. LPIPS values reinforce the PSNR ranking: MC2^2 is the only method that achieves perceptually-near-zero error (<0.02< 0.02) on every family.

MethodLaplace PSNR↑Laplace SNR↑Laplace MSE↓Laplace LPIPS↓Poisson PSNR↑Poisson SNR↑Poisson MSE↓Poisson LPIPS↓Yukawa PSNR↑Yukawa SNR↑Yukawa MSE↓Yukawa LPIPS↓Biharmonic PSNR↑Biharmonic SNR↑Biharmonic MSE↓Biharmonic LPIPS↓Helmholtz PSNR↑Helmholtz SNR↑Helmholtz MSE↓Helmholtz LPIPS↓Overall PSNR↑Overall SNR↑Overall MSE↓Overall LPIPS↓
Walk-on-Spheres (Monte Carlo)
WoS (B=8B = 8)25.2219.174.02E–030.518811.205.900.12140.822013.898.480.07930.729815.019.650.07160.731510.935.600.12470.836015.269.770.08020.7275
WoS (B=16B = 16)27.2222.172.07E–030.437714.417.110.10860.795715.5610.150.06580.668616.1111.250.06650.695812.066.730.11260.813116.9611.500.07100.6861
WoS (B=32B = 32)31.2125.171.06E–030.356114.895.590.09360.726717.5012.090.05210.625418.1213.050.06050.655313.488.140.09820.783118.9113.420.06090.6396
WoS (B=64B = 64)34.2827.185.37E–040.278215.6310.330.07720.722919.6714.260.03830.590120.4015.040.05330.610715.159.820.08280.745121.0315.540.05060.5893
WoS (B=128B = 128)37.2331.182.70E–040.209017.0112.300.06220.722722.0516.640.02730.533622.5417.180.04570.564017.0611.730.06740.750123.3117.820.04070.5370
WoS (B=256B = 256)30.2434.191.35E–040.152019.7814.470.04990.672324.1019.190.01850.472522.8919.480.03810.561419.1813.850.05300.682825.7420.240.03160.4841
WoS (B=512B = 512)43.2537.206.78E–050.106522.1316.830.03530.573327.2121.870.01190.409427.2121.850.03060.467621.4916.160.04020.600528.2922.790.02370.4315
WoS (B=1024B = 1024)46.2640.213.39E–050.070024.6519.350.02480.516130.2724.660.00840.348929.6824.310.02180.419923.9518.620.02940.544630.9425.440.01710.3800
WoS (B=2048B = 2048)49.2743.221.79E–050.041927.3122.010.01650.463032.5527.270.00480.329432.2628.240.01800.378826.5621.220.02460.486633.6728.170.01190.3299
WoS (B=4096B = 4096)52.2746.238.49E–060.012830.0824.780.01040.395435.8730.462.38E–030.239934.7029.900.01270.324129.2623.729.19E–030.368136.4630.977.91E–030.2820
WoS (B=8192B = 8192)35.2949.244.25E–060.011132.9427.646.26E–030.336738.3333.421.28E–030.193737.3031.937.75E–030.275532.0526.728.91E–030.368139.2933.805.07E–030.2375
WoS (B=16384B = 16384)58.3052.252.12E–060.003935.8630.553.57E–030.282441.5636.416.73E–040.140137.7934.415.84E–030.238434.8929.565.49E–030.311542.1437.043.13E–030.1964
WoS (B=32768B = 32768)61.3555.261.06E–060.001938.1133.511.96E–030.232944.3939.381.75E–040.104442.1536.793.80E–030.166437.7732.343.25E–030.258344.9839.491.88E–030.1592
WoS (B=65536B = 65536)64.3158.275.31E–070.001141.7836.471.04E–030.187447.9243.381.75E–040.064440.9339.032.44E–030.165440.6735.341.87E–030.210247.8042.311.11E–030.1263
WoS (B=131072B = 131072)67.3261.272.66E–070.000344.7739.475.49E–040.144250.5846.094.53E–050.046944.4941.131.57E–030.117043.5638.231.06E–030.169450.6045.311.16E–030.0974
Traditional Denoisers (applied to B=8B = 8 WoS)
GAUSSIAN23.4017.641.26E–040.502213.688.250.08920.510617.4611.920.04200.382717.6212.220.05890.398814.269.020.03340.465817.2811.950.05610.4543
GAUSSIAN (HEAVY)25.2020.232.25E–040.178813.878.440.10520.376616.9610.760.06730.296516.1010.620.03900.333614.928.920.08400.344017.7411.770.04460.2964
TOTAL VARIATION37.9832.291.86E–040.132918.1612.850.06510.398723.5017.960.01820.314323.7417.840.03180.277718.8413.600.04880.356124.3418.740.02580.2787
NON-LOCAL MEANS30.6033.911.34E–040.100420.4915.060.03740.495623.9618.350.01170.303225.1919.800.02650.339421.0615.820.03520.462426.2720.820.02270.3390
Neural Network Methods
FNO44.0638.176.05E–050.019624.2618.830.02370.334929.5824.147.15E–030.180129.4324.030.02230.262424.5019.060.02370.298830.3424.890.01590.2196
PINO18.5012.610.04990.18946.180.830.27240.42108.262.810.24110.48325.690.303.00770.24674.590.650.29340.45157.682.230.25850.3886
CoCoGen23.9812.250.48530.061811.115.880.12660.849514.679.110.06860.64756.690.003.00770.24675.890.003.00770.246716.9511.410.18320.0208
DiffusionPDE24.7719.004.16E–030.651911.115.880.12660.849514.679.110.06860.64756.690.003.00770.24675.890.003.00770.246717.2311.680.05290.6958
MC2^2 (OURS)60.2154.522.53E–060.000140.8035.373.69E–040.008546.8341.081.83E–050.001634.8829.489.15E–050.078535.8734.641.15E–050.009144.1038.052.37E–030.0208

Table 11. Full per-family results across all four metrics on the PDEZoo test set. PSNR and SNR are reported in dB; MSE and LPIPS are unitless. Arrows indicate metric direction. Bold marks the best result per column among learned and post-processing methods. MC2^2 uses a single B=8B = 8 WoS input.

Bias–Variance Analysis of PDE Solvers

Here, we formalize the bias–variance tradeoff among three estimators of an elliptic PDE solution uu: the pure Monte Carlo Walk-on-Spheres estimator u^K\hat{u}_K, a pure neural operator Φθo\Phi_{\theta}^{o}, and our hybrid MC2^2 estimator u~K=Φθ(u^K,a)\tilde{u}_K = \Phi_{\theta}(\hat{u}_K, a), where aa denotes any auxiliary conditioning information available to the corrector, such as the source term, domain mask, boundary data, or PDE parameters. Throughout, we use the standard bias–variance decomposition of mean-squared error under squared loss [11, 13]. We present the analysis pointwise at a fixed query x∈Ωx \in\Omega; the same conclusions hold for discretized fields by summing over grid points, or in L2(Ω)L^{2}(\Omega) by integration. For an estimator TT of u(x)u(x), the conditional mean-squared error decomposes as

Data fractionBiharmonic PSNR↑Biharmonic SNR↑Biharmonic MSE↓Biharmonic LPIPS↓Helmholtz PSNR↑Helmholtz SNR↑Helmholtz MSE↓Helmholtz LPIPS↓Laplace PSNR↑Laplace SNR↑Laplace MSE↓Laplace LPIPS↓Poisson PSNR↑Poisson SNR↑Poisson MSE↓Poisson LPIPS↓Yukawa PSNR↑Yukawa SNR↑Yukawa MSE↓Yukawa LPIPS↓Overall PSNR↑Overall SNR↑Overall MSE↓Overall LPIPS↓
0.04×31.6826.2760.01270.197030.7225.3105.74e−030.238650.6544.6571.26e−050.005831.0725.7123.84e−030.240135.4129.9941.26e−030.108835.7930.2784.84e−030.1594
0.08×33.5128.1020.01100.153434.6629.2573.03e−030.135853.7647.7676.19e−060.001435.0729.7091.77e−030.128539.5334.1145.13e−040.039139.1833.6633.37e−030.0929
0.16×33.9628.5550.01170.126835.3229.9083.31e−030.081454.7048.7165.09e−060.000635.9230.5551.61e−030.071440.3034.8865.36e−040.017839.9134.3923.54e−030.0607
0.32×35.4130.0038.79e−030.133335.9530.5383.14e−030.113055.3249.3304.33e−060.000536.4231.0601.71e−030.087740.5835.1665.39e−040.023840.6135.0972.92e−030.0729
0.64×35.7730.3638.57e−030.082638.9633.5561.41e−030.018358.6252.6292.18e−060.000140.1434.7816.05e−040.011944.6339.2091.49e−040.002743.4737.9522.23e−030.0239
1.0× (full)35.7430.3327.51e−030.062840.2934.8781.13e−030.010859.7653.7721.80e−060.000141.6336.2724.70e−040.005445.8640.4401.58e−040.001644.4838.9721.92e−030.0167

Table 12. Effect of training data scale on MC2^{2} performance. All models use B=8B = 8 WoS inputs. Fractions denote the proportion of the full training set used.

E[(T−u(x))2∣P]=(E[T∣P]−u(x))2+Var⁡(T∣P),(3)\mathbb{E}\left[(T-u(x))^{2}\mid P\right]=\left(\mathbb{E}[T\mid P]-u(x)\right)^{2}+\operatorname{Var}(T\mid P), \tag*{(3)}

where PP denotes the underlying PDE instance and the expectation is over the internal randomness of the estimator.

Monte Carlo WoS Solver

Let u^K(x)=1K∑k=1KYk(x)\hat{u}_{K}(x)=\frac{1}{K}\sum_{k=1}^{K}Y_{k}(x) be the WoS estimator using KK independent walks per query point. WoS expresses the solution of elliptic boundary-value problems as the expectation of a random variable induced by Green’s functions and boundary hitting distributions [27, 30]. Thus, ignoring the small ε\varepsilon-shell termination bias,

E[u^K(x)∣P]=u(x).(4)\mathbb{E}[\hat{u}_{K}(x)\mid P]=u(x). \tag*{(4)}

With ε\varepsilon-shell termination, the estimator has a small boundary bias that decays linearly in ε\varepsilon with rate depending on local geometry [26, 2]:

∣E[u^K(x)∣P]−u(x)∣=O(ε).(5)\left|\mathbb{E}[\hat{u}_{K}(x)\mid P]-u(x)\right|=\mathcal{O}(\varepsilon). \tag*{(5)}

The variance of the average of KK independent walks is

Var⁡(u^K(x)∣P)=σP2(x)K,(6)\operatorname{Var}(\hat{u}_{K}(x)\mid P)=\frac{\sigma_{P}^{2}(x)}{K}, \tag*{(6)}

where σP2(x)=Var⁡(Y1(x)∣P)\sigma_{P}^{2}(x)=\operatorname{Var}(Y_{1}(x)\mid P) is the single-walk variance. Therefore,

E[(u^K(x)−u(x))2∣P]=σP2(x)K+O(ε2).(7)\mathbb{E}\left[(\hat{u}_{K}(x)-u(x))^{2}\mid P\right]=\frac{\sigma_{P}^{2}(x)}{K}+\mathcal{O}(\varepsilon^{2}). \tag*{(7)}

Thus, pure WoS is approximately unbiased, but its variance decays only as O(1/K)\mathcal{O}(1/K).

Neural Operators

A pure neural operator maps a PDE specification directly to a solution field, Φθop:(f,g,Ω)↦u^op\Phi_{\theta}^{\mathrm{op}}:(f,g,\Omega)\mapsto\hat{u}^{\mathrm{op}}, as in neural operator and physics-informed neural operator methods [? 25]. Conditional on a PDE instance PP, the prediction is deterministic at inference time, so

Var⁡(Φθop(P)(x)∣P)=0.(8)\operatorname{Var}(\Phi_{\theta}^{\mathrm{op}}(P)(x)\mid P)=0. \tag*{(8)}

However, the estimator generally has approximation bias

bθ(P,x)=Φθop(P)(x)−u(x),(9)b_{\theta}(P,x)=\Phi_{\theta}^{\mathrm{op}}(P)(x)-u(x), \tag*{(9)}

so its conditional MSE is

E[(Φθop(P)(x)−u(x))2∣P]=bθ(P,x)2.(10)\mathbb{E}\left[\left(\Phi_{\theta}^{\mathrm{op}}(P)(x)-u(x)\right)^{2}\mid P\right]=b_{\theta}(P,x)^{2}. \tag*{(10)}

Unlike Monte Carlo error, this bias does not decrease with additional inference-time sampling compute. Moreover, under distribution shift, learned PDE solvers can incur substantially larger approximation error because their predictions are tied to the training distribution rather than to an unbiased inference-time estimator [23, 7].

MC2: Learned Correction of a Monte Carlo Estimator

MC2^2 takes a finite-budget Monte Carlo estimate and optional auxiliary PDE information as input. Let XK=(u^K,a)X_K = (\hat{u}_K, a), where aa may include the source term, domain mask, boundary information, or PDE parameters. MC2^2 outputs u~K=Φθ(XK)\tilde{u}_K = \Phi_\theta(X_K). The population-optimal corrector under squared loss is the conditional expectation, also known as the regression function or Bayes estimator under squared loss [13]:

Φ⋆(XK)(x)=E[u(x)∣XK].(11)\Phi^\star(X_K)(x) = \mathbb{E}[u(x) \mid X_K]. \tag*{(11)}

Equivalently,

Φ⋆=arg⁡min⁡ΦE[∥Φ(XK)−u∥22].(12)\Phi^\star= \arg\min_{\Phi} \mathbb{E}\left[\left\|\Phi(X_K)-u\right\|_2^2\right]. \tag*{(12)}

This immediately gives a useful comparison to raw WoS. Since the identity map Φ(XK)=u^K\Phi(X_K) = \hat{u}_K is an admissible candidate in the minimization above, the Bayes-optimal corrector is never worse than raw WoS in population MSE:

E[∥Φ⋆(XK)−u∥22]≤E[∥u^K−u∥22].(13)\mathbb{E}\left[\left\|\Phi^\star(X_K)-u\right\|_2^2\right] \le\mathbb{E}\left[\left\|\hat{u}_K-u\right\|_2^2\right]. \tag*{(13)}

If XKX_K also contains every input that a standalone neural operator would use, such that ΦθOP(P)\Phi_\theta^{\mathrm{OP}}(P) can be expressed as a measurable function of XKX_K, then the operator is also an admissible candidate in the minimization, and

E[∥Φ⋆(XK)−u∥22]≤E[∥ΦθOP(P)−u∥22].(14)\mathbb{E}\left[\left\|\Phi^\star(X_K)-u\right\|_2^2\right] \le\mathbb{E}\left[\left\|\Phi_\theta^{\mathrm{OP}}(P)-u\right\|_2^2\right]. \tag*{(14)}

Thus, the optimal corrector dominates both raw Monte Carlo and standalone neural prediction whenever the same information is available. In practice, MC2^2 learns an approximation Φθ≈Φ⋆\Phi_\theta\approx\Phi^\star, so the guarantee holds up to approximation, optimization, and finite-data error:

E[∥Φθ(XK)−u∥22]=E[∥Φ⋆(XK)−u∥22]+Eapprox(θ),(15)\mathbb{E}\left[\left\|\Phi_\theta(X_K)-u\right\|_2^2\right] = \mathbb{E}\left[\left\|\Phi^\star(X_K)-u\right\|_2^2\right] + \mathcal{E}_{\mathrm{approx}}(\theta), \tag*{(15)}

where Eapprox(θ)≥0\mathcal{E}_{\mathrm{approx}}(\theta) \ge0 captures the gap between the learned corrector and the Bayes-optimal corrector.

Variance Reduction from a Lipschitz Corrector

The preceding argument explains why an optimal corrector can reduce MSE. We now show explicitly how a learned corrector reduces Monte Carlo variance.

Assume that Φθ\Phi_\theta is LθL_\theta-Lipschitz in its first argument (holding aa fixed) on the support of low-budget Monte Carlo inputs:

∥Φθ(z,a)−Φθ(z′,a)∥2≤Lθ∥z−z′∥2.(16)\left\|\Phi_\theta(z,a)-\Phi_\theta(z',a)\right\|_2 \le L_\theta\left\|z-z'\right\|_2. \tag*{(16)}

Let u^K′\hat{u}'_K be an independent copy of u^K\hat{u}_K conditioned on the same PDE instance PP (with the same auxiliary information aa). Using the standard identity

Var⁡(Z)=12E[∥Z−Z′∥22](17)\operatorname{Var}(Z) = \frac{1}{2}\mathbb{E}\left[\left\|Z-Z'\right\|_2^2\right] \tag*{(17)}

for an independent copy Z′Z', where Var⁡\operatorname{Var} denotes total variance for vector-valued ZZ [22], we obtain

Var⁡(u~K∣P)=12E[∥Φθ(u^K,a)−Φθ(u^K′,a)∥22∣P](18)\operatorname{Var}(\tilde{u}_K \mid P) = \frac{1}{2}\mathbb{E}\left[\left\|\Phi_\theta(\hat{u}_K,a)-\Phi_\theta(\hat{u}'_K,a)\right\|_2^2 \mid P\right] \tag*{(18)}
≤Lθ22E[∥u^K−u^K′∥22∣P](19)\le\frac{L_\theta^2}{2}\mathbb{E}\left[\left\|\hat{u}_K-\hat{u}'_K\right\|_2^2 \mid P\right] \tag*{(19)}
=Lθ2Var⁡(u^K∣P),(20)= L_\theta^2\operatorname{Var}(\hat{u}_K \mid P), \tag*{(20)}

where the first equality applies the variance identity to u~K\tilde{u}_K, the inequality applies the Lipschitz bound, and the final equality applies the variance identity to u^K\hat{u}_K. Therefore, if the corrector is contractive on the Monte Carlo noise directions (Lθ<1L_\theta< 1 locally on the relevant input distribution), MC2^2 strictly reduces conditional variance:

Var⁡(u~K∣P)<Var⁡(u^K∣P).(21)\operatorname{Var}(\tilde{u}_K \mid P) < \operatorname{Var}(\hat{u}_K \mid P). \tag*{(21)}

The cost is that the corrected estimator is no longer guaranteed to be unbiased. The pointwise bias is

Bias⁡(u~K(x)∣P)=E[Φθ(u^K,a)(x)∣P]−u(x)(22)\operatorname{Bias}(\tilde{u}_K(x) \mid P) = \mathbb{E}[\Phi_\theta(\hat{u}_K,a)(x) \mid P] - u(x) \tag*{(22)}

and MC2^2 improves over raw WoS in MSE whenever the variance reduction more than compensates for the introduced squared bias:

Bias⁡2(u~K∣P)+Var⁡(u~K∣P)<Var⁡(u^K∣P).(23)\operatorname{Bias}^2(\tilde{u}_K \mid P) + \operatorname{Var}(\tilde{u}_K \mid P) < \operatorname{Var}(\hat{u}_K \mid P). \tag*{(23)}

Comparison and Asymptotic Behavior

The three estimators occupy different regions of the bias–variance plane. Pure WoS is approximately unbiased and broadly applicable, but its variance decays only as O(1/K)O(1/K), making high accuracy expensive. Pure neural operators eliminate Monte Carlo variance at inference time but introduce approximation bias that does not vanish with additional sampling compute and may grow under distribution shift [23, 7]. MC2\mathrm{MC}^2 starts from an approximately unbiased Monte Carlo estimator and applies a learned correction that reduces finite-sample variance in exchange for controlled bias.

Asymptotic consistency. As K→∞K \to\infty, the raw WoS estimator converges to the true solution up to the ε\varepsilon-shell bias. By continuity of Φθ\Phi_\theta,

u~K=Φθ(u^K,a)→Φθ(u,a),(24)\widetilde{u}_K = \Phi_\theta(\widehat{u}_K, a) \to\Phi_\theta(u, a), \tag*{(24)}

so asymptotic consistency requires the corrector to approach the identity on clean inputs: Φθ(u,a)≈u\Phi_\theta(u, a) \approx u. Training on multiple Monte Carlo budgets, including high-quality estimates, encourages this behavior heuristically by exposing the corrector to a range of noise levels. We do not prove asymptotic consistency formally; rather, our empirical results demonstrate that in the practical finite-compute regime, the variance reduction from MC2\mathrm{MC}^2 substantially outweighs the introduced bias.

References

  1. [1]Jens Berg and Kaj Nyström. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28–41, 2018.DOI
  2. [2]Ilia Binder and Mark Braverman. The rate of convergence of the Walk on Spheres algorithm. Geometric and Functional Analysis, 22(3):558–587, 2012. doi: 10.1007/s00039-012-0161-z.DOI
  3. [3]Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. Polygon Mesh Processing. CRC Press, 2010.DOI
  4. [4]Susanne Brenner and Ridgway Scott. The Mathematical Theory of Finite Element Methods. Springer, 3rd edition, 2008.
  5. [5]William L. Briggs, Van Emden Henson, and Steve F. McCormick. A Multigrid Tutorial. SIAM, 2nd edition, 2000.
  6. [6]Keenan Crane, Fernando de Goes, Mathieu Desbrun, and Peter Schröder. Digital geometry processing with discrete exterior calculus. ACM SIGGRAPH 2013 Courses, 2013.DOI
  7. [7]Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022.arxiv.org/abs/2201.05624
  8. [8]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009. doi: 10.1109/CVPR.2009.5206848.DOI
  9. [9]David Donoho. Data science at the singularity. Harvard Data Science Review, 6(1), 2024. doi: 10.1162/99608f92.b91339ef.
  10. [10]Chi-Mao Fan, Tsung-Jung Liu, and Kuan-Hsien Liu. SUNet: Swin transformer UNet for image denoising. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, May 2022. doi: 10.1109/iscas48785.2022.9937486.DOI
  11. [11]Stuart Geman, Elié Bienenstock, and René Doursat. Neural networks and the bias/variance dilemma. Neural Computation, 4(1):1–58, 1992.DOI
  12. [12]Jayesh K. Gupta and Johannes Brandstetter. Towards multi-spatiotemporal-scale generalized pde modeling. arXiv preprint arXiv:2209.15616, 2022.arxiv.org/abs/2209.15616
  13. [13]Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2 edition, 2009.
  14. [14]Maximilian Herde, Nils Wandel, Siddhartha Mishra, Nils Thuerey, and Bogdan Raonic. Poseidon: Efficient foundation models for pdes. In Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2405.19101
  15. [15]Masanobu Horie and Naoto Mitsume. Physics-embedded neural networks: Graph neural pde solvers with mixed boundary conditions. In Advances in Neural Information Processing Systems, volume 35, pages 23218–23229. Curran Associates, Inc., 2022.arxiv.org/abs/2205.11912
  16. [16]Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. Diffusionpde: Generative pde-solving under partial observation. Advances in Neural Information Processing Systems, 37: 130291–130323, 2024.arxiv.org/abs/2406.17763
  17. [17]Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically consistent and conditioned score-based generative models for forward and inverse problems. SIAM Journal on Scientific Computing, 2024.arxiv.org/abs/2312.10527
  18. [18]Zichao Jiang, Junyang Jiang, Qinghe Yao, and Gengchao Yang. A neural network-based pde solving algorithm with high precision. Scientific Reports, 13(1):4479, 2023.DOI
  19. [19]Guoliang Jin, Lin Chen, and Jiamei Dong. Monte carlo finite element method of structure reliability analysis. Reliability Engineering & System Safety, 40(1), 1993. doi: 10.1016/0951-8320(93)90121-E.
  20. [20]Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, volume 35, pages 23593–23606, 2022.arxiv.org/abs/2201.11793
  21. [21]Yuehaw Khoo, Jianfeng Lu, and Lexing Ying. Solving parametric pde problems with artificial neural networks. European Journal of Applied Mathematics, 32(3):421–435, 2021.DOI
  22. [22]Erich L Lehmann and George Casella. Theory of Point Estimation. Springer, 2 edition, 1998.
  23. [23]Zilu Li, Guandao Yang, Xi Deng, Christopher De Sa, Bharath Hariharan, and Steve Marschner. Neural caches for monte carlo partial differential equation solvers. In SIGGRAPH Asia 2023 Conference Papers, SA ’23. Association for Computing Machinery, 2023. doi: 10.1145/3610548.3618141.DOI
  24. [24]Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Iiu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=c8P9NQVtmn0.
  25. [25]Zongyi Li, Hongkai Zheng, Nikola B. Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. CoRR, abs/2111.03794, 2021. URL https://arxiv.org/abs/2111.03794.
  26. [26]Michael Mascagni and Chi-Ok Hwang. ε-Shell error analysis for “Walk On Spheres” algorithms. Mathematics and Computers in Simulation, 63(2):93–104, 2003. doi: 10.1016/S0378-4754(03)00038-7.DOI
  27. [27]M. Muller. Some continuous monte carlo methods for the dirichlet problem. Ann. Math. Statist., 27(3), 1956.DOI
  28. [28]Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations, 2017.arxiv.org/abs/1711.10561
  29. [29]Pratik Rathore, Weimu Lei, Zachary Frangella, Lu Lu, and Madeleine Udell. Challenges in training PINNs: A loss landscape perspective, 2024.arxiv.org/abs/2402.01868
  30. [30]Rohan Sawhney and Keenan Crane. Monte carlo geometry processing: A grid-free approach to pde-based methods on volumetric domains. ACM Trans. Graph., 39(4), 2020.DOI
  31. [31]Rohan Sawhney, Dario Seyb, Wojciech Jarosz, and Keenan Crane. Grid-free monte carlo for pdes with spatially varying coefficients. ACM Trans. Graph., 41(4), July 2022. ISSN 0730-0301. doi: 10.1145/3528223.3530134.DOI
  32. [32]Rohan Sawhney, Dario Seyb, Wojciech Jarosz, and Keenan Crane. Walk on stars: A grid-free monte carlo method for pdes with neumann boundary conditions. ACM Trans. Graph., 2023.DOI
  33. [33]Ryusuke Sugimoto, Terry Chen, Yiti Jiang, Christopher Batty, and Toshiya Hachisuka. A practical walk-on-boundary method for boundary value problems. ACM Transactions on Graphics, 42(4):1–16, July 2023. ISSN 1557-7368. doi: 10.1145/3592109.DOI
  34. [34]Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. In NeurIPS Datasets and Benchmarks Track, 2022.arxiv.org/abs/2210.07182
  35. [35]Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Brussels, Belgium, 2018. Association for Computational Linguistics. doi: 10.18653/v1/W18-5446. URL https://aclanthology.org/W18-5446.DOI
  36. [36]Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. SuperGLUE: A stricter benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems, volume 32, 2019.arxiv.org/abs/1905.00537
  37. [37]Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.arxiv.org/abs/2007.14527
  38. [38]Rui Zhang, Qi Meng, Rongchan Zhu, Yue Wang, Wenlei Shi, Shihua Zhang, Zhi-Ming Ma, and Tie-Yan Liu. Monte carlo neural pde solver for learning pdes via probabilistic representation, 2023.DOI

Paper details

vlnc:W_hpq2upau
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents