betaFeedback

WorksComputer Science

Sample-Efficient Control of Magnetoconvective Flows via Differentiable Simulation

Abstract

Magnetohydrodynamic (MHD) fluid flows are central to demanding engineering applications, most prominently fusion blanket cooling, where flow control is a promising tool to optimize key properties such as heat transfer. While active flow control (AFC) has recently advanced through machine learning, its application to MHD flows remains largely unexplored. In this work, we present ϕϕ-PICT, a novel differentiable GPU-accelerated MHD solver targeted at AFC. We validate ϕϕ-PICT against MHD benchmarks and demonstrate its high computational efficiency. Integrated into FluidGym, ϕϕ-PICT enables sample-efficient gradient-based algorithms to improve heat transfer by >25>25% and allows optimizers to achieve 103×10^3\times higher accuracy in inverse design than black-box methods. ϕϕ-PICT is publicly available at https://github.com/upb-sciml/phiPICT.

Introduction

Control of large-scale physical systems governed by high-dimensional, non-linear partial differential equations (PDEs) remains a fundamental challenge at the intersection of machine learning, physics, and control theory [10]. Among such systems, magnetohydrodynamics (MHD) introduces distinct control challenges driven by the interaction between an electrically conducting fluid and external magnetic field. In thermal flows, this coupling leads to so-called magnetoconvection, which fundamentally alters the flow structures and heat transfer. Liquid metal magnetoconvection is key to many engineering applications, such as cooling in nuclear-power facilities [32] or fusion reactor blankets [1, 53].

In fusion blanket duct flows, Zikanov and Listratov [51] showed that strong temperature fluctuations can be explained by wall jets destabilizing into large quasi-2D (Q2D) vortices that induce strong mixing. Subsequent works have explored passive strategies to mitigate these thermal stresses, including asymmetric heat loads [8] and modified inlet geometries [52]. However, passive strategies cannot adapt dynamically to non-stationary, evolving turbulent structures. Consequently, closed-loop active flow control (AFC) emerges as a promising candidate for real-time flow stabilization and heat transfer improvement in magnetoconvective fluid dynamics.

Data-driven approaches, particularly reinforcement learning (RL), have emerged as a promising technique for AFC and have achieved notable successes in drag reduction, heat transfer enhancement, and flow separation [31, 6, 41, 18, 28, 36]. Concurrently, differentiable simulators enable direct backpropagation through physical dynamics and can substantially improve sample-efficiency over traditional RL [5, 39]. Despite these advances, the application of RL and gradient-based control to liquid metal flows in closed channels or ducts remains largely unexplored, primarily due to the lack of differentiable, RL-compatible MHD simulators.

To bridge this gap, we introduce ϕ\phi-PICT, a GPU-accelerated, differentiable MHD solver and integrate it into the open-source FluidGym framework. In summary, our main contributions are (1) ϕ\phi-PICT Solver: A differentiable, GPU-accelerated MHD solver for low magnetic Reynolds numbers that enables end-to-end backpropagation through coupled MHD and thermal dynamics and is validated against canonical MHD and convection direct numerical simulation benchmarks; (2) FluidGym Integration: A modular 3D liquid metal Rayleigh-Bénard convection control environment in FluidGym for black-box RL and gradient-based policy learning; and (3) Empirical Evaluation: Demonstrations showing that gradient-based optimization via ϕ\phi-PICT is substantially more sample-efficient across both domains, outperforming black-box RL to increase heat transfer by >25%>25\% and achieving 3 orders of magnitude higher accuracy in inverse wall-defect detection1.

Related Work

Numerical Solvers for MHD Flows. Simulating liquid metal flows in the presence of strong magnetic fields in fusion reactors typically relies on the inductionless, quasistatic approximation with magnetic Reynolds numbers Rm≪1R_{\mathrm{m}} \ll1 [12]. Common numerical software includes epotFOAM [38], based on openFOAM [44] and specialized algorithms like Tensor-Product Thomas solvers [24].

Differentiable Solvers. Differentiable fluid solvers [21, 9, 15] have become a promising tool for general PDE control [22] and gradient-based policy learning [5, 50, 39]. Existing differentiable MHD frameworks focus on high-Rm\mathrm{R}_{\mathrm{m}} astrophysics in geometries with simpler boundary conditions than those at solid walls [35]. To the best of our knowledge, no differentiable solver exists for low-Rm\mathrm{R}_{\mathrm{m}} magnetoconvective flows in closed geometries.

Reinforcement Learning for Active Flow Control. Recently, deep RL has shown great success in closed-loop flow control. Studies range from aerodynamic drag reduction [31, 17, 28] to convective heat transfer enhancement [6]. For spatially distributed control, multi-agent reinforcement learning (MARL; see Albrecht et al., [2] for an introduction) has shown promising results in terms of generalization and sample-efficiency [18, 41, 40, 26]. In nuclear fusion research, RL has been used for magnetic control of plasma in Tokamak reactors [13, 29]. However, RL for AFC has not yet been applied to liquid metal flows present, for instance, in cooling blankets. We address this gap by introducing a differentiable MHD control benchmark integrated into FluidGym [5], an open-source plug-and-play benchmark for RL and gradient-based methods for AFC.

ϕ\phi-PICT: Differentiable Simulation of Magnetoconvective Fluid Flows

Our differentiable solver ϕ\phi-PICT for MHD fluid dynamics builds on PICT [15], which provides a differentiable PISO loop, basic boundary conditions, and the adjoint mechanism for the standard velocity and pressure linear solves. To adapt this to MHD, ϕ\phi-PICT implements ϕ\phi-PISO [27 see Appendix A for details]. Specifically, it adds (1) the electric potential Poisson solver, (2) a conservative current density formulation, (3) the Lorentz force coupling, and crucially, (4) the custom gradient functions required to backpropagate through the new PISO steps related to the electric potential, current density, and Lorentz force (see Figure 1).

Figure 1. Computation of FL\mathbf{F}_L given u\mathbf{u} in ϕ\phi-PICT. Solid and dotted lines denote explicit analytical routines and autograd, respectively.

Governing Equations. Following previous works [8], we consider the so-called inductionless or quasi-static approximation. Here, the magnetic field is considered static and not affected by the fluid [see 12 for details]. Under this formulation, the incompressible momentum and continuity equations are

∂u∂t+(u⋅∇)u=−∇p+1Re∇2u+FL+FBand∇⋅u=0.(1)\frac{\partial\mathbf{u}}{\partial t}+(\mathbf{u}\cdot\nabla)\mathbf{u}=-\nabla p+\frac{1}{\mathrm{Re}}\nabla^2\mathbf{u}+\mathbf{F}_L+\mathbf{F}_B \quad\text{and}\quad\nabla\cdot\mathbf{u}=0. \tag*{(1)}

The thermal transport and the buoyancy force based on the Boussinesq approximation are given by

∂T∂t+u⋅∇T=1RePr∇2TwithFB=−GrRe2eg(T−T0).(2)\frac{\partial T}{\partial t}+\mathbf{u}\cdot\nabla T=\frac{1}{\mathrm{Re}\mathrm{Pr}}\nabla^2T \quad\text{with}\quad\mathbf{F}_B=-\frac{\mathrm{Gr}}{\mathrm{Re}^2}e_g(T-T_0). \tag*{(2)}

Finally, the current density, charge conservation, and Lorentz force are defined as

j=−∇ϕ+u×eband∇⋅j=0withFL=Ha2Rej×eb.(3)\mathbf{j}=-\nabla\phi+\mathbf{u}\times\mathbf{e}_b \quad\text{and}\quad\nabla\cdot\mathbf{j}=0 \quad\text{with}\quad\mathbf{F}_L=\frac{\mathrm{Ha}^2}{\mathrm{Re}}\mathbf{j}\times\mathbf{e}_b. \tag*{(3)}

The non-dimensional parameters of the coupled magnetohydrodynamic and convective heat transfer system are the Reynolds number Re=UH/ν\mathrm{Re}=UH/\nu, Hartmann number Ha=B0Hσ/(ρν)\mathrm{Ha}=B_0H\sqrt{\sigma/(\rho\nu)}, Prandtl number Pr=ν/α\mathrm{Pr}=\nu/\alpha, Grashof number Gr=gβΔTH3/ν2\mathrm{Gr}=g\beta\Delta TH^3/\nu^2, and Stuart number N=Ha2/ReN=\mathrm{Ha}^2/\mathrm{Re}. Here, UU is the characteristic velocity, HH is the characteristic length scale, ν\nu is the kinematic viscosity, B0B_0 is the magnetic field magnitude, σ\sigma is the electrical conductivity, ρ\rho is the fluid density, α\alpha is the thermal diffusivity, gg is the gravitational acceleration, β\beta is the thermal expansion coefficient, T0T_0 is the reference temperature, and ΔT\Delta T is the characteristic temperature difference.

Forward Simulation. An overview of the forward simulation steps computing FL\mathbf{F}_L from u\mathbf{u} is shown in Figure 1. From the charge conservation in Equation (3), we can derive the Poisson equation for the electric potential ϕ\phi: via ∇⋅j=0⇔∇2ϕ=∇⋅(u×eb)\nabla\cdot\mathbf{j}=0\Leftrightarrow\nabla^2\phi=\nabla\cdot(\mathbf{u}\times\mathbf{e}_b). Following [27, 38], we apply the conservative Lorentz force scheme proposed by Ni et al. [30].

Gradient Computation. To enable end-to-end backpropagation through ϕ\phi-PICT, we derive analytical gradients alongside the forward simulation steps (see Figure 1). Following Franz et al. [15], ∂y\partial y denotes the gradient of the loss w.r.t. yy, and the vector-Jacobian product ∂x(∂y)=(∂/∂x)T∂y\partial x(\partial y)=(\partial/\partial x)^{\mathsf{T}}\partial y yields the contribution to ∂x\partial x via yy. Backpropagation starts from ∂FL\partial\mathbf{F}_L and reaches u\mathbf{u} through u×B\mathbf{u}\times\mathbf{B} along two paths: directly via the

face current fluxes, and indirectly via the electric potential ϕ\phi. Analytical gradients are derived for two core operations: (i) the conservative face-based current density construction, which couples adjacent cells via face flux sensitivities to yield ∂ϕ(∂j)\partial\phi\left(\partial\mathbf{j}\right) and ∂(u×B)(∂j)\partial(\mathbf{u}\times\mathbf{B})\left(\partial\mathbf{j}\right), and (ii) the linear potential solve. For the discrete linear system of the electric potential Φϕ=g\Phi\phi= g with discrete potential matrix Φ\Phi and g=∇⋅g=∇⋅(u×B)g = \nabla\cdot\mathbf{g} = \nabla\cdot(\mathbf{u}\times\mathbf{B}), we solve the adjoint system ΦT∂g=∂ϕ\Phi^{T}\partial g = \partial\phi to obtain ∂g\partial g, which gives the gradient function ∂g(∂ϕ)\partial g\left(\partial\phi\right). In contrast to naive unrolling of Conjugate Gradient (CG), solving the adjoint allows O(1)O(1) memory scaling w.r.t. linear solve iterations. Standard cell operations, such as the cross products producing ∂j\partial\mathbf{j} and finally ∂u=B×∂(u×B)\partial\mathbf{u} = \mathbf{B}\times\partial(\mathbf{u}\times\mathbf{B}), are handled by PyTorch autograd (dotted lines in Figure 1). Although B\mathbf{B} is assumed constant in this work, ϕ\phi-PICT also enables gradient backpropagation with respect to B\mathbf{B}. Detailed gradient derivations are stated in Appendix § 9.§ 3.

Solver Validation. In the following, we describe how we thoroughly evaluate our MHD solver. First, we run Hartmann, Shercliff, and Hunt flows and match the analytical velocity profiles. A grid refinement study shows how the relative errors decrease with higher resolutions. Furthermore, we compare our results to the jet detachments in a duct flow discussed by Arlt [4]. To further validate our solver for magnetoconvective flows, we run validation cases spanning various Hartmann numbers and wall conduction ratios that match results previously published by Krasnov et al. [24], see Table 7 and Table 8. To validate the correctness and informativeness of simulation gradients, we run the PyTorch gradcheck tool as well as an optimization task where we learn the Stuart number of a reference simulation via backpropagation. Together, these results show the correctness of our proposed solver in both forward and backward passes (see Appendix B for detailed results).

HaNuNuref_{\mathrm{ref}}ϵ\epsilon (%)
03.7513.8001.302
1624.4535.15013.529*
3254.4174.4400.518
4873.6213.4405.265*
6503.1813.1500.975
7963.0343.0100.812

Table 7. Experimental results for the convection in a box case [24]. *In these cases, the flow is unstable and does not converge to a steady-state solution, as pointed out by Krasnov et al. [24]. Therefore, our results do not match the reference numbers.

CwC_wNuNuref_{\mathrm{ref}}ϵ\epsilon (%)
No MHD3.3593.3310.838
03.2413.2450.139
0.013.0403.0460.183
0.12.2762.2770.050
11.5371.5270.681
501.3871.3900.184

Table 8. Experimental results for the convection in a box case [24].

Further Improvements. Due to the large computational cost associated with fluid simulations, we incorporate a series of improvements into ϕ\phi-PICT to enhance its efficiency and consequently the downstream performance of FluidGym for (magnetoconvective) environments. To fully utilize the GPU, we introduce batched simulations of over 250 parallel FluidGym environments, yielding substantial speedups over FluidGym based on PICT. To further reduce simulation runtime at scale, we introduce three core optimizations to the CG solver in ϕ\phi-PICT: (i) transferring the whole CG solve execution to the GPU without intermediate CPU communication, (ii) warm-starting the pressure solve from the previous time step, and (iii) adopting adaptable solver tolerances during intermediate iterations, following OpenFOAM. Consequently, ϕ\phi-PICT achieves a substantial 20×20\times average speedup over the baseline PICT solver in FluidGym. Notably, for some environments, speedups of up to approx. 150×150\times are achieved (see Table 2 in Appendix § 9.§ 4). Specifically for MHD, we additionally leverage preconditioning of the electric potential Poisson linear system through the algebraic multigrid (AMG; Ruge and Stüben [34]) method via PyAMG (Bell et al. [7]).

Environmentϕ\phi-PICTBB
Airfoil2D-easy×114.07\times114.0764
Airfoil2D-hard×149.56\times149.5696
Airfoil2D-medium×114.32\times114.3296
Airfoil3D-easy×2.79\times2.792
Airfoil3D-hard×1.77\times1.772
Airfoil3D-medium×2.97\times2.974
CylinderJet2D-easy×117.32\times117.32384
CylinderJet2D-hard×61.72\times61.72128
CylinderJet2D-medium×69.93\times69.93192
CylinderJet3D-easy×14.01\times14.0124
CylinderJet3D-hard×2.68\times2.682
CylinderJet3D-medium×5.57\times5.5712
CylinderRot2D-easy×129.49\times129.49384
CylinderRot2D-hard×131.05\times131.05512
CylinderRot2D-medium×85.75\times85.75192
RBC2D-easy×72.95\times72.9564
RBC2D-hard×119.09\times119.09128
RBC2D-medium×116.26\times116.26192
RBC2D-wide-easy×62.17\times62.1796
RBC2D-wide-hard×80.94\times80.9496
RBC2D-wide-medium×80.78\times80.7896
RBC3D-easy×16.13\times16.1396
RBC3D-hard×16.81\times16.8196
RBC3D-medium×16.06\times16.0696
RBC3D-wide-easy×6.79\times6.7924
RBC3D-wide-hard×7.81\times7.8124
RBC3D-wide-medium×7.62\times7.6224
TCFLarge3D-both-easy×4.06\times4.0648
TCFLarge3D-both-hard×6.03\times6.0396
TCFLarge3D-both-medium×6.16\times6.1696
TCFLarge3D-bottom-easy×3.12\times3.1232
TCFLarge3D-bottom-hard×4.25\times4.2596
TCFLarge3D-bottom-medium×4.18\times4.1896
TCFSmall3D-both-easy×6.36\times6.3696
Mean×20\times20

Table 2. Individual speedups per environment of ϕ\phi-PICT over PICT in FluidGym with optimal batch size BB, determined on a NVIDIA A100 with 80GB VRAM. Optimal batch sizes might differ on different GPUs.

Geometric mean runtime improvements of $\phi$-PICT over PICT across non-MHD FluidGym environments

Figure 2. Geometric mean runtime improvements of ϕ\phi-PICT over PICT across all non-MHD FluidGym environments. Dots indicate speedup factors for individual environments.

Learning Closed-loop Control: Liquid Metal Rayleigh-Bénard Convection

Experimental Setup

In the following, we present a novel control setup for 3D liquid metal Rayleigh-Bénard convection (RBC) in a closed box of size Lx×Ly×Lz=5×1×5L_x \times L_y \times L_z = 5 \times1 \times5 following Yanagisawa et al. [49] that is heated at the bottom and cooled from the top. The setup is visualized in Figure 3.

Uncontrolled MHD RBC setup with sensor probes, agents, electrodes, and temperature field

Figure 3. Uncontrolled MHD RBC setup with electrodes at the Hartmann walls at Ra=50Q\mathrm{Ra} = 50Q.

Motivated by the idea that quasi-two-dimensional convection rolls have desirable properties in fusion blankets, the control environment aims to stabilize these rolls. The stability of the rolls depends on the ratio of the Rayleigh number Ra\mathrm{Ra} to the Chandrasekhar number Q=Ha2Q = \mathrm{Ha}^2. When fixing QQ and increasing Ra\mathrm{Ra}, the flow exhibits decreasing magnetic damping and higher instability, which increases control difficulty. As studied by Yanagisawa et al. [49], there are distinct flow regimes one could consider regarding Ra\mathrm{Ra} in terms of QQ. Particularly, these can be characterized by (i) a transition from five to four rolls between 10Q10Q and 30Q30Q; and (ii) a four-roll oscillation and a transition from four to three rolls between 30Q30Q and 50Q50Q. Based on these findings, we consider Ra=20Q\mathrm{Ra} = 20Q and Ra=50Q\mathrm{Ra} = 50Q.

Numerical Setup. We simulate the flow with ϕ\phi-PICT, solving Equations (1)–(3) with a uniform horizontal magnetic field eb=ex\mathbf{e}_b = \mathbf{e}_x and gravity eg=−ey\mathbf{e}_g = -\mathbf{e}_y. We use Pr=0.025\mathrm{Pr} = 0.025 and Q=103Q = 10^3. All walls are no-slip and electrically insulating. The bottom and top plates are isothermal at T=1\mathrm{T} = 1 and T=0\mathrm{T} = 0, with T0=0.5T_0 = 0.5, and the side walls are adiabatic. The magnetic field damps velocity gradients along eb\mathbf{e}_b, so the rolls align with xx and are stacked along zz, and the walls at x=0x = 0 and x=Lxx = L_x are the Hartmann walls, i.e., the walls perpendicular to the magnetic field. The grid has 160×32×160160 \times32 \times160 cells, with tanh refinement towards all walls. Along xx, the grading resolves the Hartmann layers of thickness δ∼H/Ha\delta\sim H/\mathrm{Ha} with at least three cells. The time step adapts to a CFL number of 0.80.8 with control step length Tcontrol=1.5T_{\mathrm{control}} = 1.5 free-fall time units. Discrete control steps are denoted by index kk. A set of 30 initial states for training, validation, and testing is obtained by evolving the conduction profile with small temperature perturbations for 30 free-fall times. At each reset, we randomize the roll configuration by applying a random combination of mirroring, small temperature noise, and a short random warm-up similar to the procedures in FluidGym (Becktepe et al. [5]).

Actuation, Observations, Reward. Following Becktepe et al. [5], we formalize the control task as a partially observable stochastic game (POSG) for multi-agent reinforcement learning (MARL) based on Albrecht et al. [2]. In particular, we consider MM individual agents acting in the fluid domain. While all agents share the same policy, each agent with index i∈[0,…,M−1]i \in[0,\ldots,M-1] acts locally via aia_i and receives an individual reward rir_i and observation oio_i. Here, each agent corresponds to one of M=10M = 10 electrode stations distributed along zz on both Hartmann walls. Each station holds two electrodes along yy, so agent ii sets two normalized currents ai∈[−1,1]2a^i \in[-1,1]^2. Opposing electrodes on both walls carry the same current, keeping the forcing symmetric about x=Lx/2x = L_x/2. Since the box is otherwise insulating, the joint action is projected onto zero net current before being applied. A zero action thus recovers the uncontrolled flow. Each agent observes 960 velocity and temperature sensors on the full xx–yy grid, within a local z-window that covers its own station and its neighbours’ actuators, padded beyond the side walls with zero velocity and the conduction-profile temperature. The global reward at control step kk balances heat transfer enhancement against electrical power consumption:

rk=NukNuref−1−λPel,kPref,r_k = \frac{\mathrm{Nu}_k}{\mathrm{Nu}_{\mathrm{ref}}} - 1 - \lambda\frac{P_{\mathrm{el},k}}{P_{\mathrm{ref}},}

where both terms are averaged over the inner solver steps of step kk. Here, Nu=1+RaPr⁡⟨uy(T−T0)⟩V\mathrm{Nu}=1+\sqrt{\mathrm{Ra}}\operatorname{Pr}\langle u_y(T-T_0)\rangle_V is the volume-averaged Nusselt number. Quantities indexed with ref denote the uncontrolled baseline. The injected electrical power Pel=N∫Seϕwjin dSP_{\mathrm{el}}=N\int_{S_e}\phi_w j_{\mathrm{in}}\,\mathrm{d}S over electrode surface SeS_e is normalized by the thermal driving power Pref=∫Vuy(T−T0) dVP_{\mathrm{ref}}=\int_V u_y(T-T_0)\,\mathrm{d}V. Following Vignon et al. [41], individual rewards rki=β(Nuki/Nuref−1)+(1−β)rkr_k^i=\beta(\mathrm{Nu}_k^i/\mathrm{Nu}_{\mathrm{ref}}-1)+(1-\beta)r_k blend local and global quantities.

Control Algorithms. As a black-box RL algorithm we consider Soft Actor-Critic (SAC; [19]) implemented in the Stable-Baselines3 (SB3; [33]) framework using its default hyperparameters. To leverage the differentiable simulation, we train policies using backpropagation through time (BPTT/APG; [45, 16]) and Short-Horizon Actor-Critic (SHAC; [48]). In both approaches, policy parameters are updated by backpropagating the gradient of the discounted return over a horizon of 10 control steps through the physics simulation. Notably, the naive BPTT algorithm does not leverage explicit exploration, value function estimation, or experience replay. All algorithms are trained across 5 random seeds on the FluidGym train split. After training, policies are evaluated deterministically on 10 test episodes. Detailed hyperparameters are stated in Appendix § 11.§ 2.

Results

To account for the 3×3\times computational overhead of activation checkpointing and backward passes for differentiable methods, we measure training progress in forward-equivalent simulation steps. Figure 4 shows the mean training reward trajectories. Notably, BPTT shows the fastest convergence and reaches its final reward after ≈1 000\approx1\,000 forward-equivalent steps, corresponding to ≈333\approx333 environment steps. SHAC and SAC perform similarly at Ra=20Q\mathrm{Ra}=20\mathrm{Q}, while SHAC slightly outperforms SAC at Ra=50Q\mathrm{Ra}=50\mathrm{Q}. However, neither SAC nor SHAC reaches the reward achieved by BPTT throughout 10k forward-equivalent training steps. Notably, despite lacking SHAC’s stabilization mechanisms, BPTT proves more effective here, demonstrating the sample-efficiency enabled by ϕ\phi-PICT. This is consistent with the final evaluation of policies on the test set (see Table 1). Over 10 test episodes, BPTT achieves the highest mean reward across all methods and regimes. Qualitative temperature fields generated by the final BPTT policy are shown in Figure 5. Notably, for the Ra=20Q\mathrm{Ra}=20\mathrm{Q} regime, BPTT is able to stabilize five convection rolls, improving heat transfer by over 25% throughout testing. For the Ra=50Q\mathrm{Ra}=50\mathrm{Q} regime, quantitative results show more than a 15% increase in Nu\mathrm{Nu}. However, due to the higher Rayleigh number and associated turbulence, stabilizing the system becomes substantially harder in this regime. These findings align with the final temperature fields controlled by BPTT, where four instead of five rolls are stabilized by the controller compared to the Ra=20Q\mathrm{Ra}=20\mathrm{Q} regime.

Figure 4. Mean training reward per step over forward-equivalent control steps for BPTT, SAC, and SHAC (accounting for the ∼3×\sim3\times step cost of backpropagation and activation checkpointing). Solid lines and shaded areas represent mean ±\pm standard deviation across seeds, respectively.

MHD-RBC temperature fields stabilized by the final BPTT policies

Figure 5. MHD-RBC temperature fields stabilized by the final BPTT policies.

Ra\mathrm{Ra}SACBPTTSHAC
20Q0.1330.2360.146
50Q−0.0180.1160.033

Table 1. Interquartile mean reward over 10 test episodes across regimes (best method in bold).

When inspecting the actions applied by the final BPTT policy, we observe current injection related to the five convection rolls. Figure 6 shows the wall potential at the Hartmann wall at x=0x=0, where actuators are visualized through gray rectangles. Notably, we observe five large patches of alternating positive and negative electric potential spanning the wall from z=0z=0 to z=5z=5, likely injecting current in a way that produces a Lorentz force stabilizing the convection rolls.

Figure 6. BPTT policy producing five convection rolls, visualized through the wall potential ϕw\phi_w.

These results further suggest that control policies learned through the differentiable physics systematically leverage the underlying equations to stabilize the system and improve its overall heat transfer. Detailed qualitative and quantitative results across all algorithms are stated in Appendix § 12.

Inverse Design: Detecting Defective Wall Patches in a Convection Box

Experimental Setup

In the following, we consider an inverse design problem in a 3D buoyant liquid metal flow in a cube ([37]). Motivated by the idea that cracks in the insulating coatings of fusion blankets can locally short-circuit the electric currents, the task aims to detect such effects from sparse velocity measurements in the bulk of the flow (see Figure 7). The walls are divided into 24 patches, each with its own wall conductance ratio CwC_w, and the task is to recover this CwC_w map solely from a small number of velocity probes in the bulk of the flow. Each candidate map is evaluated by simulating the flow and comparing its probe signals with those produced by the hidden ground truth. In addition to black-box optimization, the differentiability of ϕ\phi-PICT provides the gradient of this mismatch with respect to all patches, which enables gradient-based optimization.

Figure 7. Defective wall patch detection setup. Based on sparse sensor probes, the goal is to recover defective wall patches in a convective box.

Flow Configuration. The flow setup follows the convection benchmark of Tagawa et al. [37], subject to a horizontal magnetic field eb=ex\mathbf{e}_b = \mathbf{e}_x and gravity eg=−ey\mathbf{e}_g = -\mathbf{e}_y. The dimensionless parameters are set to Pr=0.0321\mathrm{Pr} = 0.0321, Gr=3.115×106\mathrm{Gr} = 3.115 \times10^6, and Ha=100\mathrm{Ha} = 100. Simulations are conducted in the unit cube of size Lx×Ly×Lz=1×1×1L_x \times L_y \times L_z = 1 \times1 \times1. A hot wall with constant temperature T=0.5T = 0.5 is prescribed at z=−0.5z = -0.5, a cold wall with T=−0.5T = -0.5 is set at z=0.5z = 0.5. Temperatures are relative to a reference temperature T0=0T_0 = 0, and the remaining walls are adiabatic. All walls are no-slip. Using the thin-wall condition, a fixed CwC_w is prescribed per patch. The grid has 64 cells per dimension, i.e., 64×64×6464 \times64 \times64 cells in total, with tanh refinement (see Krasnov et al. [24] for details) towards all walls. We use a fixed time step of Δt=2×10−4\Delta t = 2 \times10^{-4} free-fall time units. All runs start from the same statistically steady flow ustart\mathbf{u}_{\mathrm{start}} obtained with a uniform Cw=0.1C_w = 0.1.

Parameterization and Target. Each of the six cube surfaces is equally divided into 2×22 \times2 patches, resulting in 24 design parameters p=(log⁡10Cw0,…,log⁡10Cw23)\mathbf{p} = (\log_{10} C_w^0, \ldots, \log_{10} C_w^{23}), bounded by Cw∈[10−3,50]C_w \in[10^{-3}, 50]. A random seed defines the hidden ground-truth map: Each face receives a base value log⁡10Cw∼U(−2,0)\log_{10} C_w \sim U(-2, 0) shared by its four patches, and each patch is independently marked as defective with probability 0.25, which raises its log⁡10Cw\log_{10} C_w by an offset drawn from U(0.5,1.5)U(0.5, 1.5). Simulating this map for N=100N = 100 steps from ustart\mathbf{u}_{\mathrm{start}} yields the target flow utarget\mathbf{u}_{\mathrm{target}}. Starting from random maps drawn from the same distribution, the optimizers have no prior information about defect locations.

Sensors and Objective. The flow is observed only through 64 point probes, each measuring all three velocity components in a single cell, i.e., 192 measurements in total. The probe cells are drawn once, uniformly at random from the core of the domain at a distance of at least 0.1 from all walls, and are kept fixed across all seeds and optimizers. With the sensor operator O\mathcal{O}, each evaluation of a candidate p\mathbf{p} restarts from an initial, steady-state velocity field ustart\mathbf{u}_{\mathrm{start}}, simulates the same NN steps to obtain uN(p)\mathbf{u}_N(\mathbf{p}), and computes the objective measuring the deviation from the target measurements:

J(p)=∥O(uN(p))−O(utarget)∥22∥O(ustart)−O(utarget)∥22.(4)J(\mathbf{p}) = \frac{\left\|\mathcal{O}(\mathbf{u}_N(\mathbf{p}))-\mathcal{O}(\mathbf{u}_{\mathrm{target}})\right\|_2^2}{\left\|\mathcal{O}(\mathbf{u}_{\mathrm{start}})-\mathcal{O}(\mathbf{u}_{\mathrm{target}})\right\|_2^2}. \tag*{(4)}

Optimization Algorithms. For this task, we consider the black-box optimization algorithm CMA-ES [20] with an initial step size of σ0=0.5\sigma_0 = 0.5 and its default population size as a baseline. Further, we evaluate the gradient-based L-BFGS-B algorithm [11] implemented in SciPy [42]. An optimization run stops after reaching J≤10−5J \le10^{-5} or after a wall-time limit of 30 minutes. Detailed hyperparameters are stated in Appendix § 11.§ 3.

Results

In the following, we discuss the optimization results of CMA-ES and L-BFGS-B on the defective detection task. The optimization progress over time is shown in Figure 8. We compare the methods in terms of the objective JJ over the wall-clock time. Dots mark where the median reaches the stop objective J=10−5J = 10^{-5}. Notably, within the given time budget, the gradient-based L-BFGS-B reaches an objective value approximately 3 orders of magnitude smaller than CMA-ES. Similarly, the RMSE between the optimized and ground-truth wall patch conductivities is substantially smaller for the gradient-based optimizer. These results demonstrate the efficiency of gradient-based optimization via differentiable simulation for inverse problems.

Defect detection optimization using CMA-ES and L-BFGS-B

Figure 8. Defect detection optimization using CMA-ES and L-BFGS-B: best objective JJ over wall-clock time. Lines and shading show median and min–max range across seeds, respectively.

When inspecting the final obtained wall patch conductivities by the optimizers, we observe notable differences. Figure 9 shows the ground-truth CwC_w maps per wall as well as the respective errors of the best configuration returned by each optimizer. For the walls at ±x\pm x, both optimizer errors are comparably low. However, for the remaining four walls, CMA-ES exhibits substantial errors, whereas the errors of L-BFGS-B remain small. Most importantly, for the walls ±x\pm x and ±z\pm z, the errors of L-BFGS-B are negligible. Together, these results highlight that differentiating through ϕ\phi-PICT yields informative gradients, which enable gradient-based optimizers to recover the true underlying defective patches accurately. Across additional seeds, residual errors vary across walls without favoring specific coordinates, which indicates this localized variation is not systematic.

Figure 9. Final wall conductance ratio maps for CMA-ES and L-BFGS-B (seed 0). We visualize the ground-truth CwC_w maps per cube wall and the respective errors of the final results of both optimizers.

Similarly, the final velocity fields generated with the optimized CwC_w maps differ in the errors with respect to the ground-truth velocity field. As shown in Figure 10, the velocity field produced by the CMA-ES CwC_w map shows relative errors on the order of 10%, while the L-BFGS-B velocity field errors are negligible. This comparison demonstrates that the solution obtained through differentiable physics is robust and accurately recovers the underlying physical properties of the target flow.

Figure 10. Ground-truth velocity fields and relative errors of velocity fields produced by the final CwC_w maps optimized via CMA-ES and L-BFGS-B.

Limitations & Future Work

In this work, we present ϕ\phi-PICT, a differentiable GPU-accelerated solver for liquid metal MHD flows targeted at control and inverse design problems. Despite its GPU acceleration and efficiency optimizations, computational limitations remain. Currently, the solver requires a CUDA-capable GPU and is restricted to single-GPU execution, though scaling to multi-GPU architectures presents a viable direction for future research. Furthermore, high-resolution 3D simulations remain computationally demanding and highlight the need for surrogate modeling approaches.

Beyond addressing the aforementioned limitations, ϕ\phi-PICT can serve as an efficient data generator to train fast surrogate models, ranging from machine learning surrogates to physics foundation models ([46, 3, 23]) and possibly future models targeted at controlled flows. While combining RL and differentiable physics has proven highly sample-efficient in robotics ([48, 47]), its application to fluids remains largely unaddressed, a gap our benchmark aims to bridge. Finally, we plan to extend this framework to more control and inverse problems spanning both hydrodynamics and magnetohydrodynamics.

Generative AI Use

Generative AI tools were used for implementation, code optimization, and refactoring, as well as for improving the written text of the manuscript. The authors reviewed and verified all AI-assisted content and take responsibility for the final content of the paper.

Acknowledgments

JB and SP acknowledge funding from the European Research Council (ERC Starting Grant “Ko-OpeRaDE”) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No. 101161457). The computations were performed on the compute cluster of the Lamarr Institute for Machine Learning and Artificial Intelligence, as well as on the high-performance computers “Noctua 2” and “Otus” at the NHR Center Paderborn Center for Parallel Computing (PC2), both of which are funded by the Federal Ministry of Research, Technology and Space and by the state of North Rhine-Westphalia. The work of DK and JS is supported by the Deutsche Forschungsgemeinschaft.

Differentiable Simulation of Magnetoconvective Fluid Flows

Governing Equations

Adapting the formulation of Mas de les Valls [27], we extend the PISO algorithm implemented in PICT (Franz et al. [15]) to ϕ\phi-PISO. From the charge conservation in Equation (3), we can derive the Poisson equation for the electric potential ϕ\phi:

∇⋅j=0⇔∇⋅(−∇ϕ+u×eb)=0⇔−∇⋅∇ϕ+∇⋅(u×eb)=0⇔∇2ϕ=∇⋅(u×eb).(5)\begin{aligned} \nabla\cdot\mathbf{j} = 0 \\ &\Leftrightarrow\nabla\cdot(-\nabla\phi+ \mathbf{u} \times\mathbf{e}_b) = 0 \\ &\Leftrightarrow-\nabla\cdot\nabla\phi+ \nabla\cdot(\mathbf{u} \times\mathbf{e}_b) = 0 \\ &\Leftrightarrow\nabla^2 \phi= \nabla\cdot(\mathbf{u} \times\mathbf{e}_b). \tag*{(5)} \end{aligned}

Boundary Conditions For the electric potential ϕ\phi, boundary conditions are defined by the wall conductance ratio

Cw=σwτwσH(6)C_w = \frac{\sigma_w \tau_w}{\sigma H} \tag*{(6)}

with

Cw=0⟹∂ϕ∂n∣w=0(Perfectly insulating wall)(7)C_w = 0 \Longrightarrow\left.\frac{\partial\phi}{\partial n}\right|_w = 0 \qquad\text{(Perfectly insulating wall)} \tag*{(7)}
Cw→∞⟹ϕ=const.(Perfectly conducting wall)(8)C_w \to\infty\Longrightarrow\phi= \text{const.} \qquad\text{(Perfectly conducting wall)} \tag*{(8)}

As formulated by Walker [43], for walls with a constant conduction ratio Cw>0C_w > 0 and wall thickness τ≪H\tau\ll H, we employ the so called thin-wall boundary condition

∂ϕ∂n∣w=Cw∇τ2ϕ∣w.(9)\left.\frac{\partial\phi}{\partial n}\right|_w = C_w \left.\nabla_{\tau}^{2}\phi\right|_w. \tag*{(9)}

Finite Volume Formulation

Notation For consistency and direct compatibility with the underlying solver, we adopt the finite volume method (cf. Maliska [25]) notation used by Franz et al. [15], summarized below:

Current cell PP

Set of valid neighbor cells of PP F∈FF \in\mathbf{F}

Set of faces between PP and FF f∈ff \in\mathbf{f}

Set of (virtual) boundary neighbor cells of PP B∈BB \in\mathbf{B}

Set of boundary-faces between PP and BB b∈bb \in\mathbf{b}

Set of diagonal neighbors ’tangential’ to FF DF∈DFD_F \in\mathbf{D}_F

Set of diagonal boundary faces ’tangential’ to bb Db∈DbD_b \in\mathbf{D}_b

Sign of logical face direction on computational grid NfN_f

Evaluated/value at current cell [□]P[\square]_P or □P\square_P

Evaluated/value at neighbor cell [□]F[\square]_F or □F\square_F

(Linear) Interpolation □‾\overline{\square} Following this convention, for cell PP (index ii) and neighbor cell F∈FF \in\mathbf{F} (index jj), the entries of a discretized operator matrix Φ\mathbf{\Phi} are indexed as:

Diagonal entry at current cell[ΦP]P=ΦiiOff-diagonal entry for neighbor[ΦF]P=ΦijNeighbor row, current column[ΦP]F=ΦjiNeighbor row diagonal entry[ΦF]F=Φjj\begin{aligned} \text{Diagonal entry at current cell} \quad[\Phi_P]_P &= \Phi_{ii} \\ \text{Off-diagonal entry for neighbor} \quad[\Phi_F]_P &= \Phi_{ij} \\ \text{Neighbor row, current column} \quad[\Phi_P]_F &= \Phi_{ji} \\ \text{Neighbor row diagonal entry} \quad[\Phi_F]_F &= \Phi_{jj} \end{aligned}

Potential Solve To solve the Poisson equation for the electric potential (∇2ϕ∗=∇⋅(u×B)\nabla^2\phi^* = \nabla\cdot(\mathbf{u} \times\mathbf{B})), we leverage the Conjugate Gradient (CG) solver in PICT. The discrete potential matrix Φ\Phi is constructed similarly to the discrete pressure matrix PP, but the diagonal entries A−1A^{-1} are omitted. This leaves us with the Laplace coefficients [α‾jj]f[\overline{\alpha}_{jj}]_f:

[ΦF]P=[α‾jj]f[ΦP]P=∑f−[α‾jj]f.(10)\begin{aligned} [\Phi_F]_P &= [\overline{\alpha}_{jj}]_f \\ [\Phi_P]_P &= \sum_f -[\overline{\alpha}_{jj}]_f . \tag*{(10)} \end{aligned}

Non-orthogonal grid corrections follow the same formulation as the pressure equation.

Applying the divergence theorem, the right-hand side of the potential equation is discretized as

[∇⋅(u×B)]P≈∑f(u×B)f⋅nfaf.[\nabla\cdot(\mathbf{u} \times\mathbf{B})]_P \approx\sum_f (\mathbf{u} \times\mathbf{B})_f \cdot\mathbf{n}_f a_f .

In the generalized coordinate systems, this can be expressed as

[∇⋅(u×B)]P=∑F→j[JTj⋅(u×B)]fNF.(11)[\nabla\cdot(\mathbf{u} \times\mathbf{B})]_P = \sum_{F\rightarrow j} [\mathbf{J}\mathbf{T}_j \cdot(\mathbf{u} \times\mathbf{B})]_f N_F . \tag*{(11)}

Face-based Lorentz Force To ensure discrete charge conservation, we compute the Lorentz force from face-centered current density fluxes X\mathcal{X} following the conservative scheme of Ni et al. [30]:

Xfj=−[α‾jj]fNF[ϕ]F−[ϕ]P⏟[∇ϕ]f+[JTj⋅(u×B)]f.(12)\mathcal{X}_f^j = -[\overline{\alpha}_{jj}]_f N_F \underbrace{[\phi]_F - [\phi]_P}_{[\nabla\phi]_f} + [\mathbf{J}\mathbf{T}_j \cdot(\mathbf{u} \times\mathbf{B})]_f . \tag*{(12)}

The cell-centered current density jP\mathbf{j}_P is obtained by averaging opposite face fluxes along each coordinate direction jj and transforming to the physical domain:

jP=1J∑j(T−1).j(12∑F→jXfj).(13)\mathbf{j}_P = \frac{1}{J}\sum_j (\mathbf{T}^{-1})_{.j}\left(\frac{1}{2}\sum_{F\rightarrow j}\mathcal{X}_f^j\right). \tag*{(13)}

The resulting Lorentz body force is then evaluated as FL=N(jP×B)\mathbf{F}_L = \mathbf{N}(\mathbf{j}_P \times\mathbf{B}).

Thin-Wall Approximation For walls with arbitrary conductance ratio CwC_w, we apply the thin-wall boundary condition. For a fluid cell PP adjacent to a boundary face b∈bb \in\mathbf{b}, this condition couples the potential to the tangential neighbor cells DF∈DFD_F \in\mathbf{D}_F. For each tangential direction τ≠n\tau\ne n, we define the surface Laplace coefficient as:

στ,f=Cw⋅12([αττ]P∣Tn,P−1∣+[αττ]F∣Tn,F−1∣).(14)\sigma_{\tau,f} = C_w \cdot\frac{1}{2}\left([\alpha_{\tau\tau}]_P|\mathbf{T}^{-1}_{n,P}| + [\alpha_{\tau\tau}]_F|\mathbf{T}^{-1}_{n,F}|\right). \tag*{(14)}

where αττ\alpha_{\tau\tau} denotes the orthogonal Laplace coefficient along τ\tau. The discrete potential matrix Φ\Phi in Equation (10) is updated accordingly:

[ΦP]P←[ΦP]P−στ,f[\Phi_P]_P \leftarrow[\Phi_P]_P-\sigma_{\tau,f}
[ΦDF]P←[ΦDF]P+στ,f.(15)[\Phi_{D_F}]_P \leftarrow[\Phi_{D_F}]_P+\sigma_{\tau,f}. \tag*{(15)}

while the wall-normal face contribution remains zero as in the insulating case. For cells bordered by multiple thin-wall boundaries (such as corner cells), these corrections accumulate additively across all adjacent boundary faces b∈bb \in\mathbf{b}.

Gradients

For gradient computation, we adopt the notation of Franz et al. [15]. In this work, ∂y:=(∂L∂y)T\partial y := \left(\frac{\partial L}{\partial y}\right)^{\mathsf{T}} denotes the adjoint, i.e., the gradient of a scalar loss LL with respect to yy, rather than a differential. For a function y=h(x)y=h(x), the vector-Jacobian product ∂x(∂y)f=(∂f∂x)T∂y\partial x(\partial y)_f = \left(\frac{\partial f}{\partial x}\right)^{\mathsf{T}}\partial y yields the contribution to ∂x\partial x that flows through yy and has the same shape as xx. If xx influences the loss through multiple quantities, the individual contributions are summed to obtain ∂x\partial x. We denote the function by hh to avoid confusion with the face index ff. Standard operations handled automatically by PyTorch’s autograd are omitted.

Local Formulation. xx and yy denote arbitrary physical quantities in the computational graph, e.g., the potential ϕ\phi or the velocity u\mathbf{u}. As in Franz et al. [15], we write them per cell or face: [ϕ]P[\phi]_P is considered a scalar and [u]P[\mathbf{u}]_P a vector of three components, although the corresponding fields are vectors over all cells. Since every operation, except for the linear solves, only couples a cell to its neighbors, the global Jacobian ∂h/∂x\partial h/\partial x is sparse and never assembled. Instead, we write the vector-Jacobian product for a single cell PP. Since [x]P[x]_P only enters quantities of PP itself and of its neighbors F∈FF \in\mathcal{F}, all other terms of the global product vanish. For a cell-based quantity yy, we have

∂[x]P(∂y)=(∂[y]P∂[x]P)T[∂y]P+∑F∈F(∂[y]F∂[x]P)T[∂y]F,(16)\partial[x]_P(\partial y)=\left(\frac{\partial[y]_P}{\partial[x]_P}\right)^{\mathsf{T}}[\partial y]_P+\sum_{F\in\mathcal{F}}\left(\frac{\partial[y]_F}{\partial[x]_P}\right)^{\mathsf{T}}[\partial y]_F, \tag*{(16)}

and analogously, for a face-based quantity yy, a sum over the bounding faces f∈Ff \in\mathcal{F}:

∂[x]P(∂y)=∑f∈F(∂[y]f∂[x]P)T[∂y]f.(17)\partial[x]_P(\partial y)=\sum_{f\in\mathcal{F}}\left(\frac{\partial[y]_f}{\partial[x]_P}\right)^{\mathsf{T}}[\partial y]_f. \tag*{(17)}

Inserting the local forward rule for yy turns this into an explicit per-cell expression that maps directly onto a GPU kernel.

Overview of the Backward Pass. The velocity u\mathbf{u} affects the Lorentz force through u×B\mathbf{u}\times\mathbf{B} along two paths: directly via the face current flux, and indirectly via the electric potential ϕ\phi, whose Poisson equation has the right-hand side g=∇⋅(u×B)g=\nabla\cdot(\mathbf{u}\times\mathbf{B}). Starting from the downstream gradient ∂FL\partial F_L, the backward pass proceeds as follows:

  1. Lorentz force: ∂j=Ha2ReB×∂FL\partial\mathbf{j}=\frac{\mathrm{Ha}^{2}}{\mathrm{Re}}\mathbf{B}\times\partial F_L.

  1. Cell current density to face fluxes: ∂j→∂χfj\partial\mathbf{j}\to\partial\chi_f^j (Eq. (18)).

  1. Face fluxes to potential: ∂Xfj→∂ϕ\partial\mathcal{X}^{j}_{f} \to\partial\phi (Eq. (22)).

  1. Face fluxes to u×B\mathbf{u} \times\mathbf{B}, direct path: ∂Xfj→∂(u×B)\partial\mathcal{X}^{j}_{f} \to\partial(\mathbf{u} \times\mathbf{B}) (Eq. (23)).

  1. Potential to u×B\mathbf{u} \times\mathbf{B}, indirect path: adjoint solve ΦT∂g=∂ϕ\Phi^{T}\partial\mathbf{g} = \partial\phi, followed by the backward divergence ∂g→∂(u×B)\partial\mathbf{g} \to\partial(\mathbf{u} \times\mathbf{B}) (Paragraph Potential Solve).

  1. Both contributions to ∂(u×B)\partial(\mathbf{u} \times\mathbf{B}) from steps 4 and 5 are summed.

  1. Cross product: ∂u=B×∂(u×B)\partial\mathbf{u} = \mathbf{B} \times\partial(\mathbf{u} \times\mathbf{B}).

The following paragraphs derive steps 2 – 5 in detail.

Face-based Lorentz Force. We trace the analytical backward pass starting from the Lorentz body force FL=Ha2Re(j×B)\mathbf{F}_{L} = \frac{\mathrm{Ha}^{2}}{\mathrm{Re}}(\mathbf{j} \times\mathbf{B}). Backpropagating through the face-averaged current density in Equation (13) yields the face flux sensitivity:

∂Xfj(∂j)=∑i(12J(T−1)ij∂ji).(18)\partial\mathcal{X}^{j}_{f}(\partial\mathbf{j}) = \sum_{i} \left(\frac{1}{2J}(\mathbf{T}^{-1})_{ij}\partial\mathbf{j}^{i}\right). \tag*{(18)}

The cell gradient [∂j]P[\partial\mathbf{j}]_{P} is thus distributed across all bounding faces f∈Ff \in\mathcal{F} via ∂Xfj\partial\mathcal{X}^{j}_{f}.

Substituting the potential difference xfj=[ϕ]F−[ϕ]Px^{j}_{f} = [\phi]_{F} - [\phi]_{P}, the face current flux formulation (see Equation (12)) simplifies to

Xfj=−[αˉjj]fNfxfj+[JTj⋅(u×B)]f.(19)\mathcal{X}^{j}_{f} = -[\bar{\alpha}_{jj}]_{f}N_{f}x^{j}_{f} + \left[\mathbf{J}\mathbf{T}_{j}\cdot(\mathbf{u}\times\mathbf{B})\right]_{f}. \tag*{(19)}

with sensitivity with respect to xfjx^{j}_{f}:

∂xfj(∂Xfj)=−[αˉjj]fNf∂Xfj.(20)\partial x^{j}_{f}\left(\partial\mathcal{X}^{j}_{f}\right) = -[\bar{\alpha}_{jj}]_{f}N_{f}\partial\mathcal{X}^{j}_{f}. \tag*{(20)}

Because the flux at face ff couples both cell PP and neighbor FF, the individual derivatives are:

∂[ϕ]P(∂Xfj)=−∂xj(∂Xfj)=[αˉjj]fNf∂Xfj∂[ϕ]F(∂Xfj)=+∂xj(∂Xfj)=−[αˉjj]fNf∂Xfj.(21)\begin{aligned} \partial[\phi]_{P}\left(\partial\mathcal{X}^{j}_{f}\right) &= -\partial x_{j}\left(\partial\mathcal{X}^{j}_{f}\right) = [\bar{\alpha}_{jj}]_{f}N_{f}\partial\mathcal{X}^{j}_{f} \\ \partial[\phi]_{F}\left(\partial\mathcal{X}^{j}_{f}\right) &= +\partial x_{j}\left(\partial\mathcal{X}^{j}_{f}\right) = -[\bar{\alpha}_{jj}]_{f}N_{f}\partial\mathcal{X}^{j}_{f}. \tag*{(21)} \end{aligned}

Consequently, each face ff receives two gradient contributions:

  1. from the cell’s current density gradient [∂j]P[\partial\mathbf{j}]_{P}, denoted as [∂Xfj]P:=∂Xfj([∂j]P)\left[\partial\mathcal{X}^{j}_{f}\right]_{P} := \partial\mathcal{X}^{j}_{f}([\partial\mathbf{j}]_{P}), and

  1. from neighbor cell FF’s current density gradient [∂j]F[\partial\mathbf{j}]_{F}, denoted as [∂Xfj]F:=∂Xfj([∂j]F)\left[\partial\mathcal{X}^{j}_{f}\right]_{F} := \partial\mathcal{X}^{j}_{f}([\partial\mathbf{j}]_{F}).

Accounting for the reversed face normal sign (Nf′=−NfN'_{f} = -N_{f}) from neighbor FF, the total potential gradient for cell PP is:

∂[ϕ]P(∂Xfj)=∑F→j[∂[ϕ]P([∂Xfj]P)+∂[ϕ]P([∂Xfj]F)]=∑F→j[[αˉjj]fNf[∂Xfj]P+(−[αˉjj]fNf′[∂Xfj]F)]=∑F→j[αˉjj]fNf([∂Xfj]P+[∂Xfj]F).(22)\begin{aligned} \partial[\phi]_{P}\left(\partial\mathcal{X}^{j}_{f}\right) &= \sum_{F\to j}\left[\partial[\phi]_{P}\left(\left[\partial\mathcal{X}^{j}_{f}\right]_{P}\right) + \partial[\phi]_{P}\left(\left[\partial\mathcal{X}^{j}_{f}\right]_{F}\right)\right] \\ &= \sum_{F\to j}\left[[\bar{\alpha}_{jj}]_{f}N_{f}\left[\partial\mathcal{X}^{j}_{f}\right]_{P} + \left(-[\bar{\alpha}_{jj}]_{f}N'_{f}\left[\partial\mathcal{X}^{j}_{f}\right]_{F}\right)\right] \\ &= \sum_{F\to j}[\bar{\alpha}_{jj}]_{f}N_{f}\left(\left[\partial\mathcal{X}^{j}_{f}\right]_{P} + \left[\partial\mathcal{X}^{j}_{f}\right]_{F}\right). \tag*{(22)} \end{aligned}

Following the definition of [□]f\left[\Box\right]_f, we get

[JTj⋅(u×B)]f=[JTj⋅(u×B)]P+[JTj⋅(u×B)]F2.\left[\mathrm{JT}_j \cdot(\mathbf{u} \times\mathbf{B})\right]_f = \frac{\left[\mathrm{JT}_j \cdot(\mathbf{u} \times\mathbf{B})\right]_P + \left[\mathrm{JT}_j \cdot(\mathbf{u} \times\mathbf{B})\right]_F}{2}.

This leads us to the gradient function

∂[(u×B)]P(∂Xfj)=12[JTj]P∂Xfj.\partial\left[(\mathbf{u} \times\mathbf{B})\right]_P\left(\partial\mathcal{X}_f^j\right) = \frac{1}{2}\left[\mathrm{JT}_j\right]_P \partial\mathcal{X}_f^j.

Accumulating contributions from adjacent cells gives the total gradient:

∂[(u×B)]P(∂Xfj)=∑F→j12[JTj]P([∂Xfj]P+[∂Xfj]F).(23)\partial\left[(\mathbf{u} \times\mathbf{B})\right]_P\left(\partial\mathcal{X}_f^j\right) = \sum_{F \to j} \frac{1}{2}\left[\mathrm{JT}_j\right]_P\left(\left[\partial\mathcal{X}_f^j\right]_P + \left[\partial\mathcal{X}_f^j\right]_F\right). \tag*{(23)}

The complete gradients ∂[ϕ]P(∂j)\partial[\phi]_P(\partial_j) and ∂[(u×B)]P(∂j)\partial[(\mathbf{u} \times\mathbf{B})]_P(\partial_j) follow directly by substituting the face sensitivities (see Equation (18)) into Equation (22) and Equation (23), respectively.

Potential Solve Because the discrete potential matrix Φ\Phi depends purely on static grid geometry, it requires no gradient backpropagation. Following Franz et al. [15], given the downstream potential gradient ∂ϕ\partial\phi, we compute ∂g\partial g (∂ϕ\partial\phi) by solving the linear adjoint system for the potential Poisson equation (see Equations (5) and (10)):

ΦT∂g=∂ϕ,(24)\Phi^{T}\partial g = \partial\phi, \tag*{(24)}

where g=∇⋅(u×B)g = \nabla\cdot(\mathbf{u} \times\mathbf{B}). Differentiating through the discretized divergence operator in Equation (11) then yields the gradient with respect to u×B\mathbf{u} \times\mathbf{B}:

∂(u×B)P(∂g)=∑F→j[JTj∂g]fNF.(25)\partial(\mathbf{u} \times\mathbf{B})_P(\partial g) = \sum_{F \to j} \left[\mathrm{JT}_j\partial g\right]_f N_F. \tag*{(25)}

The remaining gradient pathways back to the velocity field u\mathbf{u} are handled by PyTorch’s autograd.

Further Improvements

To assess the computational efficiency of ϕ\phi-PICT and quantify individual solver improvements, we benchmark throughput across the non-MHD environments in FluidGym. For each environment, we measure throughput in environment steps per second across different configurations: unbatched PICT, ϕ\phi-PICT without batching, leave-one-out feature ablations, and full ϕ\phi-PICT across a sweep of batch sizes. The optimal batch size for ϕ\phi-PICT is automatically determined as the throughput-maximizing size that satisfies a peak GPU memory threshold (i.e., staying within a predefined percentage of device memory). Ablation variants are subsequently evaluated at this optimal batch size. The resulting speedups over original PICT, evaluated across multiple environments, are summarized in Table 2.

Linear Upwind Advection Scheme For the MHD flows studied in this work, we found the linear upwind differencing scheme for the advection term to be more stable than the previously implemented central difference scheme (see e.g. Maliska, [25]).

Potential Solve Preconditioning To accelerate CG convergence, we precondition the system using Algebraic Multigrid (AMG; [34]) via PyAMG ([7]) for the elec tric potential. Since the discrete potential matrix Φ\Phi depends only on the grid, the preconditioning can efficiently be computed once and re-used throughout the simulation. Furthermore, AMG preconditioning is applied to decrease the number of CG iterations of the adjoint solve in the backward pass.

Intermediate Pressure Tolerance In line with OpenFOAM [44], we integrate a lower intermediate tolerance for the first pressure correction step in combination with the previous final tolerance value2.

Activation Checkpointing To enable backpropagation of reward gradients through long horizons of, for instance, 25 or RL steps, we implement activation checkpointing. This allows users to trade off computational runtime and memory requirements. In our experiments, forward and backward passes through segments of 10 PISO steps can be computed on a GPU with 80 GB VRAM.

Solver Validation

We validate our differentiable MHD implementation across canonical flow configurations with known analytical solutions, following standard validation protocols from prior MHD solver literature [27, 38, 24, 14]. These benchmarks evaluate the solver’s accuracy under varying Hartmann numbers, wall conductivity ratios, and thermal boundary conditions.

Forward Validation

In the following, we discuss the validation of ϕ\phi-PICT against canonical MHD cases. We denote spatial coordinates as xx, yy, and zz and spatial averaging over a direction xx is written as ⟨⋅⟩x\langle\cdot\rangle_x.

Hartmann Flow

The Hartmann flow is a 2D flow between two infinite walls separated by a distance 2H2H exposed to a magnetic field of strength B0B_0 perpendicular to the walls. To maintain a constant mean streamwise velocity, we dynamically update the pressure gradient G=−dpdxG = -\frac{dp}{dx} depending on the difference between the current and target velocity, i.e.,

Gn←Gn−1+c(U−⟨ux⟩x,y)G^n \leftarrow G^{n-1} + c(U - \langle u_x \rangle_{x,y})

with c=0.1c = 0.1. The analytical solution for the velocity profile is given as

ux=Ucosh⁡(Ha)−cosh⁡(Hay/H)cosh⁡(Ha)−sinh⁡(Ha)/Hau_x = U \frac{\cosh(\mathrm{Ha}) - \cosh(\mathrm{Ha} y/H)}{\cosh(\mathrm{Ha}) - \sinh(\mathrm{Ha})/\mathrm{Ha}}

with reference velocity UU. Following Mas de les Valls [27], we estimate the solver accuracy using the mean absolute error (MAE). We additionally normalize by the peak analytical velocity

ϵMAE=⟨∣ux−ux,analytical∣⟩y,zmax⁡(ux,analytical)⋅100,\epsilon_{\mathrm{MAE}} = \frac{\langle|u_x - u_{x,\mathrm{analytical}}| \rangle_{y,z}}{\max(u_{x,\mathrm{analytical}})} \cdot100,

For our experiments, we set Re=10\mathrm{Re} = 10 and H=1H = 1.

Figure 11. Hartmann case: Analytical and numerical velocity profiles for different Hartmann numbers. Numerical results are averaged over the streamwise (xx) direction.

Ha\mathrm{Ha}Nx\mathrm{N}_xNy\mathrm{N}_yEy\mathrm{E}_yϵMAE\epsilon_{\mathrm{MAE}}
0.0010050100.04
5.0010050100.02
10.0010050100.03
20.0010050100.08
20.00200100100.02
20.00400200100.01

Table 3. Experimental results for the Hartmann case.

Shercliff and Hunt Flows

CaseHa\mathrm{Ha}Re\mathrm{Re}
I209
II10010
III1000100

Table 4. Experimental setup for Shercliff and Hunt cases.

Ha\mathrm{Ha}Nx\mathrm{N}_xNy\mathrm{N}_yNz\mathrm{N}_zEy\mathrm{E}_yEz\mathrm{E}_zϵMAE\epsilon_{\mathrm{MAE}}ϵRMSE\epsilon_{\mathrm{RMSE}}
20504040200500.220.48
100504060200500.310.65
1000505050200507.9810.97
100050100100200502.072.75
1000100150150200500.750.94

Table 5. Validation results for the Shercliff case.

Ha\mathrm{Ha}NxN_xNyN_yNzN_zEyE_yEzE_zϵMAE\epsilon_{\mathrm{MAE}}ϵRMSE\epsilon_{\mathrm{RMSE}}
20504040200500.150.28
100504060200500.110.31
1000505050200500.110.94
100050100100200500.030.22
1000100150150200500.020.10

Table 6. Validation results for the Hunt case.

For the Shercliff and Hunt cases, the analytical profile is given by a Fourier series

ux,analytical=UxνGH2u_{x,\mathrm{analytical}}=\frac{U_x}{\nu}GH^2
Ux=∑k=0∞2(−1)kcos⁡(αkξ)lαk3(1−U2,k−U3,k)U_x=\sum_{k=0}^{\infty}\frac{2(-1)^k\cos(\alpha_k\xi)}{l\alpha_k^3}(1-U_{2,k}-U_{3,k})
U2,k=(Cwr2,k+1−exp⁡(−2r2,k)1+exp⁡(−2r2,k))exp⁡(−r1,k(1−η))+exp⁡(−r1,k(1+η))21+exp⁡(−2r1,k)2CwNk+1+exp⁡(−2(r1,k+r2,k))1+exp⁡(−2r2,k)U_{2,k}=\frac{\left(C_w r_{2,k}+\frac{1-\exp(-2r_{2,k})}{1+\exp(-2r_{2,k})}\right)\frac{\exp(-r_{1,k}(1-\eta))+\exp(-r_{1,k}(1+\eta))}{2}}{\frac{1+\exp(-2r_{1,k})}{2}C_wN_k+\frac{1+\exp(-2(r_{1,k}+r_{2,k}))}{1+\exp(-2r_{2,k})}}
U3,k=(Cwr1,k+1−exp⁡(−2r1,k)1+exp⁡(−2r1,k))exp⁡(−r2,k(1−η))+exp⁡(−r2,k(1+η))21+exp⁡(−2r2,k)2CwNk+1+exp⁡(−2(r1,k+r2,k))1+exp⁡(−2r1,k)U_{3,k}=\frac{\left(C_w r_{1,k}+\frac{1-\exp(-2r_{1,k})}{1+\exp(-2r_{1,k})}\right)\frac{\exp(-r_{2,k}(1-\eta))+\exp(-r_{2,k}(1+\eta))}{2}}{\frac{1+\exp(-2r_{2,k})}{2}C_wN_k+\frac{1+\exp(-2(r_{1,k}+r_{2,k}))}{1+\exp(-2r_{1,k})}}
Nk=Ha2+4αk2N_k=\sqrt{\mathrm{Ha}^2+4\alpha_k^2}
r1,k,r2,k=12(±Ha+Nk)r_{1,k},r_{2,k}=\frac{1}{2}(\pm\mathrm{Ha}+N_k)
αk=(k+12)πl,\alpha_k=\left(k+\frac{1}{2}\right)\frac{\pi}{l},

with η=z/D\eta=z/D, ξ=y/H\xi=y/H, and l=H/Dl=H/D. For the Shercliff case, we set L=2L=2, H=D=1H=D=1, and Cw=0C_w=0. For the Hunt case, we set Cw=0.1C_w=0.1 for the walls at y=±Hy=\pm H.

For the Shercliff and Hunt cases, we additionally consider the cell-volume-weighted relative RMSE, denoted as

ϵRMSE=∑iVi(ux,i−ux,analytical,i)2∑iVi∑iViux,analytical,i2∑iVi⋅100,\epsilon_{\mathrm{RMSE}}= \frac{\sqrt{\frac{\sum_i V_i\left(u_{x,i}-u_{x,\mathrm{analytical},i}\right)^2}{\sum_i V_i}}} {\sqrt{\frac{\sum_i V_i u_{x,\mathrm{analytical},i}^2}{\sum_i V_i}}}\cdot100,

where ViV_i denotes the volume of cell ii and the sums run over all cells in the domain.

Figure 12. Shercliff case: Analytical and numerical velocity profiles for different Hartmann numbers. Numerical results are averaged over the streamwise (xx) direction.

Figure 13. Shercliff case: Velocity for different Hartmann numbers.

Figure 14. Hunt case: Analytical and numerical velocity profiles for different Hartmann numbers. Numerical results are averaged over the streamwise (xx) direction.

Figure 15. Hunt case: Velocity for different Hartmann numbers.

Jet Detachments in a Duct Flow with Re=2000Re = 2000 and Ha=2000Ha = 2000.

To validate the solver in a high-HaHa magnetohydrodynamic regime, we simulate Hunt flow in a rectangular duct following Krasnov et al. [24] and Arlt [4]. The flow domain has dimensions Lx×Ly×Lz=5.0×2.0×2.0L_x \times L_y \times L_z = 5.0 \times2.0 \times2.0 discretized on a grid of Nx×Ny×Nz=300×140×200N_x \times N_y \times N_z = 300 \times140 \times200. The grid is refined towards the walls along yy and zz. Periodic boundary conditions are applied along the streamwise xx-direction. The flow is governed by Re=2000Re = 2000 and Ha=2000Ha = 2000, with an external magnetic field oriented along the zz-axis. The duct walls parallel to the magnetic field are electrically insulated (Cw,∥=0C_{w,\parallel} = 0), whereas the perpendicular (Hartmann) walls have a finite conductance ratio (Cw,⊥=0.03C_{w,\perp} = 0.03). Under these conditions, the fully developed regime exhibits characteristic high-velocity side-layer jets and jet detachment phenomena as described by Arlt [4] and Krasnov et al. [24] (see Figure 16 and Figure 17).

Figure 16. Jet detachments in a duct flow with Re=2000\mathrm{Re}=2000 and Ha=2000\mathrm{Ha}=2000.

Figure 17. Jet detachments represented by the streamwise velocity ux/Uu_x/U over the duct length x/Hx/H in a duct flow with Re=2000\mathrm{Re}=2000 and Ha=2000\mathrm{Ha}=2000.

Convection in a Tall Box

Following the benchmark configuration in Krasnov et al. [24], we consider natural convection of liquid metal (mercury) in a tall box with dimensions Lx×Ly×Lz=1×7.5×1L_x \times L_y \times L_z = 1 \times7.5 \times1. Gravity acts in the negative yy-direction, while an external uniform magnetic field is applied along the horizontal xx-axis. Constant temperatures are applied at the vertical walls perpendicular to zz (T=0.5T = 0.5 at the hot wall, T=−0.5T = -0.5 at the cold wall), with all other walls thermally and electrically insulated. The flow is simulated at Gr=3×107\mathrm{Gr} = 3 \times10^{7} (Ra=7.5×105\mathrm{Ra} = 7.5 \times10^{5}) across varying magnetic field strengths spanning 0≤Ha≤7960 \le\mathrm{Ha} \le796.

As summarized in Figure 18, our computed Nusselt numbers (Nu\mathrm{Nu}) show strong agreement with the reference values across most regimes (ϵ<1.4%\epsilon< 1.4\%). At intermediate values (Ha=162\mathrm{Ha} = 162 and Ha=487\mathrm{Ha} = 487), the flow is inherently unsteady and subject to transitions between multiple flow states [24], leading to higher relative deviations.

Figure 18. Convection case I: Temperature and velocity arrows for different Hartmann numbers.

Convection in a Square Box

The second configuration evaluates horizontally driven natural convection of a fluid in a unit cubic cavity (Lx×Ly×Lz=1×1×1L_x \times L_y \times L_z = 1 \times1 \times1) at Gr=3.15×106\mathrm{Gr} = 3.15 \times10^{6} (Ra=105\mathrm{Ra} = 10^{5}) and Ha=100\mathrm{Ha} = 100 [24]. While maintaining the same orthogonal orientation of gravity (−y-y), magnetic field (xx), and temperature gradient (zz), this setup benchmarks the effect of finite wall electrical conductivity by varying the uniform wall conductance ratio Cw∈[0,50]C_w \in[0, 50], alongside a hydrodynamic baseline (No MHD).

As shown in Table 8, our steady-state results closely match the reference data across all conductance ratios, with relative errors remaining below 0.84%.

Figure 19. Convection case II: Temperature and velocity arrows for different wall conductance ratios.

Backward Validation

Similar to the PISO steps in PICT [cf. 15], we use the PyTorch gradcheck tool to validate the ϕ\phi-PICT gradient operations. Additionally, we perform a learning task where we learn a physical parameter through the differentiable MHD simulation (see Figure 20) on three previously discussed flow settings. The goal is to learn the Stuart number NN of a reference simulation given the final velocity field after a fixed number of steps, where the loss function is the MSE between the velocity of the optimized and the reference simulation. Based on the loss with respect to the reference, a backward pass through the differentiable MHD simulation is performed, and the learned Stuart number N^\hat{N} is updated using SGD. The procedure is repeated until the loss is below 10−510^{-5}.

Figure 20. Stuart number NN learning results for validating solver gradients. Gray dashed line indicates the Stuart number of the reference simulation.

Experiments

Hardware and Software Configuration

Reproducibility and Data Availability. All validation, RL, and inverse design results are publicly available in our HuggingFace dataset3. Further, all scripts to run the experiments presented in this work are available on GitHub4.

General Experimental Setup. Experiments were conducted on the following types of nodes with the following specifications: A100-40GB nodes:

  • Python: 3.12.13

  • PyTorch: 2.10.0+cu128

  • CUDA: 12.8

  • System Memory: 32 GB RAM

  • CPU: 32 cores of an AMD EPYC 7763

  • GPU: 1×1 \times NVIDIA A100 (40 GB)

A100-80GB nodes:

  • Python: 3.12.13

  • PyTorch: 2.10.0+cu128

  • CUDA: 12.8

  • System Memory: 32 GB RAM

  • CPU: 32 cores of an AMD EPYC 7742

  • GPU: 1×1 \times NVIDIA A100 (80 GB)

All SAC runs were computed on A100-40GB nodes and BPTT and SHAC experiments were conducted on A100-80GB nodes.

Control Algorithm Hyperparameters

HyperparameterValue
Policy Network TypeMlpPolicy
Policy Network Layers[256, 256]
Critic Network Layers[256, 256]
Learning Rate3×10−43 \times10^{-4}
Discount Factor (γ\gamma)0.99
Soft Update Coefficient (τ\tau)0.005
Replay Buffer Size5×1045 \times10^{4}
Batch Size256
Learning Starts100
Training Frequency1
Gradient Steps per Update−1 (equal to train_freq)
Entropy Coefficient (α\alpha)auto
Target Entropyauto
Target Update Interval1
DeviceCUDA

Table 9. SAC hyperparameters used in all RBC-MHD experiments.

HyperparameterValue
Policy Network Layers[128, 64, 32]
Hidden Layer NormalizationLayerNorm
Hidden ActivationELU
Output ActivationTanh
OptimizerAdamW
Learning Rate10−310^{-3}
Discount Factor (γ\gamma)0.999
Horizon10
Reward Scale1.0
Max. Gradient Norm0.5
Observation NormalizationTrue
Detach ObservationsTrue
Checkpoint Segment Size5
DeviceCUDA

Table 10. BPTT hyperparameters used in all RBC-MHD experiments.

HyperparameterValue
Actor network layers[256, 256]
Critic network layers[256, 256]
Hidden layer normalizationLayerNorm
Hidden activationELU
Initial Log-Std. (log⁡σ0\log\sigma_0)−2.3
Tanh action clampingTrue
Observation normalizationTrue
OptimizerAdam
Actor learning rate3×10−43 \times10^{-4}
Critic learning rate3×10−43 \times10^{-4}
Learning rate scheduleConstant
Discount factor (γ\gamma)0.99
Horizon10
Critic methodTD(λ\lambda)
TD parameter (λ\lambda)0.95
Critic iterations16
Critic mini-batches1
Target critic coefficient (α\alpha)0.2
Max. gradient norm1.0
Reward scale1.0
Checkpoint segment size5
DeviceCUDA

Table 11. SHAC hyperparameters used in all RBC-MHD experiments.

Inverse Design Optimizer Hyperparameters

HyperparameterValue
Implementationpycma
Parametrizationlog⁡10Cw\log_{10} C_w per patch
Initial step size (σ0\sigma_0)0.5
Population size4+⌊3ln⁡n⌋4 + \lfloor3 \ln n \rfloor (default)
Bounds on log⁡10Cw\log_{10} C_w[−3,log⁡1050][-3, \log_{10} 50]
Stopping objective10−510^{-5}
Wall-time limit1800 s
GradientsNone

Table 12. CMA-ES hyperparameters for the inverse CwC_w problem. Unlisted options keep the defaults of the pycma library.

HyperparameterValue
ImplementationSCIPY.OPTIMIZE
Parametrizationlog⁡10Cw\log_{10} C_w per patch
Function tolerance (FTOL)10−1210^{-12}
Gradient tolerance (GTOL)10−810^{-8}
Max. iterations15000
Max. function evaluations15000
Bounds on log⁡10Cw\log_{10} C_w[−3,log⁡1050][-3, \log_{10} 50]
Stopping objective10−510^{-5}
Wall-time limit1800 s
GradientsDifferentiable solver

Table 13. L-BFGS-B hyperparameters for the inverse CwC_w problem. Gradients are computed by the differentiable solver. Unlisted options keep the SciPy defaults.

Results

Learning Closed-loop Control: Liquid Metal Rayleigh-Bénard Convection

Figure 21. Qualitative visualizations of the final MHD RBC temperature fields controlled by the BPTT policy.

In addition to the overall test set reward comparison, we also consider the mean improvement in NuNu and the relative actuation power consumption. These are denoted in Table 14 and Table 15, respectively. Notably, while SAC achieves a greater heat transfer improvement, it also yields higher actuation power. BPTT, on the contrary, achieves a much lower power consumption at lower Nusselt improvement, while still achieving the highest overall reward, which balances NuNu improvement and actuation power.

RaRaSACBPTTSHAC
20Q+33.56%+28.59%+31.60%
50Q+10.73%+16.44%+21.18%

Table 14. Interquartile mean relative improvement in NuNu over the uncontrolled baseflow over 10 test episodes.

RaRaSACBPTTSHAC
20Q2015.72%476.94%1558.45%
50Q1309.66%477.40%1884.74%

Table 15. Interquartile mean relative actuation power increase over the uncontrolled baseflow over 10 test episodes. Lower is better.

Box plots of per-step test set rewards for SAC, BPTT, and SHAC across the $Ra = 20Q$ and $Ra = 50Q$ regimes

Figure 22. Per-step test set rewards for all algorithms across both regimes.

Inverse Design: Detecting Defective Wall Patches in a Convection Box

Defect detection optimization using CMA-ES and L-BFGS-B

Figure 23. Defect detection optimization using CMA-ES and L-BFGS-B. Left: best objective JJ over wall-clock time. Right: RMSE between true CwC_w and optimizer CwC_w map for best objective JJ. Lines and shading show median and min–max range across seeds, respectively.

References

  1. [1]G. Aiello, J. Aubert, N. Jonquères, A. Li Puma, A. Morin, and G. Rampal. Development of the Helium Cooled Lithium Lead blanket for DEMO. Fusion Engineering and Design, 89, 2014.DOI
  2. [2]Stefano V. Albrecht, Filippos Christians, and Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. 2024.
  3. [3]Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators. In Advances in Neural Information Processing Systems 37, Vancouver, BC, Canada, 2024. Neural Information Processing Systems Foundation, Inc. (NeurIPS).arxiv.org/abs/2402.12365
  4. [4]Thomas Arlt. Analyse des Stabilitätsverhaltens magnetohydrodynamischer Kanalströmungen. PhD thesis, Karlsruher Institut für Technologie (KIT), 2018.DOI
  5. [5]Jannis Becktepe, Aleksandra Franz, Nils Thuerey, and Sebastian Peitz. Plug-and-play benchmarking of reinforcement learning algorithms for large-scale flow control. In Forty-third International Conference on Machine Learning, 2026.arxiv.org/abs/2601.15015
  6. [6]Gerben Beintema, Corbetta Alessandro, Biferale Luca, and Federico Toschi. Controlling Rayleigh–Bénard convection via reinforcement learning. Journal of Turbulence, 21:585–605, 2020. ISSN null. doi: 10.1080/14685248.2020.1797059.DOI
  7. [7]Nathan Bell, Luke N. Olson, Jacob Schroder, and Ben Southworth. PyAMG: Algebraic multigrid solvers in python. Journal of Open Source Software, 8, 2023.
  8. [8]Ivan Belyaev, Dmitry Krasnov, Yuri Kolesnikov, Dmitry Biryukov, Denis Chernysh, Oleg Zikanov, and Yaroslav Listratov. Effects of symmetry on magnetohydrodynamic mixed convection flow in a vertical duct. Physics of Fluids, 32, 2020.DOI
  9. [9]Deniz A. Bezgin, Aaron B. Buhendwa, and Nikolaus A. Adams. JAX-Fluids 2.0: Towards HPC for differentiable CFD of compressible two-phase flows. Computer Physics Communications, 2025. doi: 10.1016/j.cpc.2024.109433.DOI
  10. [10]Steven L. Brunton, Bernd R. Noack, and Petros Koumoutsakos. Machine Learning for Fluid Mechanics. Annual Review of Fluid Mechanics, 52, 2020.
  11. [11]Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16, 1995.
  12. [12]P. A. Davidson. An Introduction to Magnetohydrodynamics. Cambridge University Press, 2001.
  13. [13]Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling, Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier Sauter, Cristian Sommariva, Stefano Coda, Basil Duval, Ambrogio Fasoli, Pushmeet Kohli, Koray Kavukcuoglu, Demis Hassabis, and Martin Riedmiller. Magnetic control of Tokamak plasmas through deep reinforcement learning. Nature, 602, 2022.DOI
  14. [14]Rupert W. Eardley-Brunt, Aleksander Dubas, and Andrew Davis. On scalable liquid-metal MHD solvers for fusion breeder blanket multiphysics applications. Plasma Physics and Controlled Fusion, 66, 2024.
  15. [15]Aleksandra Franz, Hao Wei, Luca Guastoni, and Nils Thuerey. PICT–A differentiable, GPU-accelerated multi-block PISO solver for simulation-coupled learning tasks in fluid dynamics. Journal of Computational Physics, 544, 2026.DOI
  16. [16]C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, Virtual, 2021.arxiv.org/abs/2106.13281
  17. [17]Xavier Garcia, Arnau Miró, Pol Suárez, Francisco Alcántara-Ávila, Jean Rabault, Bernat Font, Oriol Lehmkuhl, and Ricardo Vinuesa. Deep-reinforcement-learning-based separation control in a two-dimensional airfoil. International Journal of Heat and Fluid Flow, 116, 2025.arxiv.org/abs/2502.16993
  18. [18]Luca Guastoni, Jean Rabault, Philipp Schlatter, Hossein Azizpour, and Ricardo Vinuesa. Deep reinforcement learning for turbulent drag reduction in channel flows. The European Physical Journal E, 46, 2023.arxiv.org/abs/2301.09889
  19. [19]Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, 2018.arxiv.org/abs/1801.01290
  20. [20]Nikolaus Hansen and Andreas Ostermeier. Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation, 9, 2001.pubmed.ncbi.nlm.nih.gov/11382355
  21. [21]Philipp Holl and Nils Thuerey. ΦFlow (PhiFlow): Differentiable simulations for PyTorch, Tensor-Flow and jax. In International Conference on Machine Learning. PMLR, 2024.
  22. [22]Philipp Holl, Nils Thuerey, and Vladlen Koltun. Learning to control pdes with differentiable physics. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.arxiv.org/abs/2001.07457
  23. [23]Benjamin Holzschuh, Qiang Liu, Georg Kohl, and Nils Thuerey. PDE-transformer: Efficient and versatile transformers for physics simulations. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 2025.arxiv.org/abs/2505.24717
  24. [24]Dmitry Krasnov, Ali Akhtari, Oleg Zikanov, and Jörg Schumacher. Tensor-product-Thomas elliptic solver for liquid-metal magnetohydrodynamics. Journal of Computational Physics, 474, 2023.DOI
  25. [25]Clovis R. Maliska. Fundamentals of Computational Fluid Dynamics: The Finite Volume Method, volume 135 of Fluid Mechanics and Its Applications. Springer International Publishing, 2023.
  26. [26]Thorben Markmann, Michiel Straat, Sebastian Peitz, and Barbara Hammer. Control of Rayleigh-Bénard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime. arXiv preprint arXiv:2504.12000, 2025.
  27. [27]Elisabet Mas de les Valls. Development of a Simulation Tool for MHD Flows under Nuclear Fusion Conditions. PhD thesis, Universitat Politècnica de Catalunya, 2011.
  28. [28]Ricard Montalà, Bernat Font, Pol Suárez, Jean Rabault, Oriol Lehmkuhl, Ricardo Vinuesa, and Ivette Rodriguez. Flow separation control of an infinite wing section via multi-agent reinforcement learning. Journal of Fluid Mechanics, 1040, 2026.
  29. [29]Antoine Mouchamps, Arthur Malherbe, Adrien Bolland, and Damien Ernst. Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research. Software Impacts, 27, 2026.
  30. [30]Ming-Jiu Ni, Ramakanth Munipalli, Peter Huang, Neil B. Morley, and Mohamed A. Abdou. A current density conservative scheme for incompressible MHD flows at a low magnetic Reynolds number. Part II: On an arbitrary collocated mesh. Journal of Computational Physics, 227, 2007.
  31. [31]Jean Rabault, Miroslav Kuchta, Atle Jensen, Ulysse Réglade, and Nicolas Cerardi. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of Fluid Mechanics, 865, 2019.
  32. [32]V. I. Rachkov, A. P. Sorokin, and A. V. Zhukov. Thermal Hydraulic Studies of Liquid-Metal Coolants in Nuclear-Power Facilities. High Temperature, 56, 2018.
  33. [33]Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-Baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22, 2021.
  34. [34]J. W. Ruge and K. Stüben. 4. Algebraic Multigrid. In Multigrid Methods, pages 73–130. Society for Industrial and Applied Mathematics, 1987.DOI
  35. [35]Leonard Storcks, Nils Thuerey, and Tobias Buck. Per Astronomix ad Astra: High-Order Differentiable (Magneto)hydrodynamics with Energy-Conserving Self-Gravity. arXiv preprint arXiv:2607.18176, 2026.arxiv.org/abs/2607.18176
  36. [36]Michiel Straat, Thorben Markmann, Sebastian Peitz, and Barbara Hammer. Improving the robustness of control of chaotic convective flows with domain-informed reinforcement learning. In 2026 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2026.arxiv.org/abs/2511.00272
  37. [37]Toshio Tagawa, Guillaume Authié, and René Moreau. Buoyant flow in long vertical enclosures in the presence of a strong horizontal magnetic field. Part 1. Fully-established flow. European Journal of Mechanics - B/Fluids, 21, 2002.DOI
  38. [38]Alessandro Tassone. Magnetic induction and electric potential solvers for incompressible MHD flows. In Proceedings of CFD with OpenSource Software, 2016.DOI
  39. [39]Matteo Tomasetto, Nicolò Botteghi, Gabriele Bruni, and Andrea Manzoni. Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems, 2026.arxiv.org/abs/2607.16177
  40. [40]Joel Vasanth, Jean Rabault, Francisco Alcántara-Ávila, Mikael Mortensen, and Ricardo Vinuesa. Multi-agent Reinforcement Learning for the Control of Three-Dimensional Rayleigh–Bénard Convection. Flow, Turbulence and Combustion, 2024.DOI
  41. [41]Colin Vignon, Jean Rabault, Joel Vasanth, Francisco Alcántara-Ávila, Mikael Mortensen, and Ricardo Vinuesa. Effective control of two-dimensional Rayleigh–Bénard convection: Invariant multi-agent reinforcement learning is all you need. Physics of Fluids, 35, 2023.
  42. [42]Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, Ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimmrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, António H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in python. Nature Methods, 17, 2020.
  43. [43]John S. Walker. Magnetohydrodynamic flows in rectangular ducts with thin conducting walls. Part I: Constant area and variable area ducts with strong uniform magnetic fields. Journal de Mécanique, 20:79–112, 1981.
  44. [44]H. G. Weller, G. Tabor, H. Jasak, and C. Fureby. A tensorial approach to computational continuum mechanics using object-oriented techniques. Computers in Physics, 12, 1998.DOI
  45. [45]P.J. Werbos. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE, 78, 1990.DOI
  46. [46]Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for PDEs on general geometries. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research. PMLR, 2024.arxiv.org/abs/2402.02366
  47. [47]Eliot Xing, Vernon Luk, and Jean Oh. Stabilizing reinforcement learning in differentiable multi-physics simulation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.DOI
  48. [48]Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, and Miles Macklin. Accelerated Policy Learning with Parallel Differentiable Simulation. In International Conference on Learning Representations, 2021.arxiv.org/abs/2204.07137
  49. [49]Takatoshi Yanagisawa, Yozo Hamano, Takehiro Miyagoshi, Yasuko Yamagishi, Yuji Tasaka, and Yasushi Takeda. Convection patterns in a liquid metal under an imposed horizontal magnetic field. Physical Review E, 2013.pubmed.ncbi.nlm.nih.gov/24483570
  50. [50]Pietro Zanotta, Dibakar Roy Sarkar, Honghui Zheng, Somdatta Goswami, and Jan Drgona. CINOC: Cardinality-invariant neural operator policies for scalable PDE control. In Forty-Third International Conference on Machine Learning, 2026.arxiv.org/abs/2605.25867
  51. [51]Oleg Zikanov and Yaroslav Listratov. Numerical investigation of MHD heat transfer in a vertical round tube affected by transverse magnetic field. Fusion Engineering and Design, 113, 2016.DOI
  52. [52]Oleg Zikanov, Dmitry Krasnov, Thomas Boeck, and Semion Sukoriansky. Decay of turbulence in a liquid metal duct flow with transverse magnetic field. Journal of Fluid Mechanics, 867, 2019.
  53. [53]Milan Zmitko, Noël Thomas, Antonella LiPuma, Laurent Forest, Laurence Cogneau, Jörg Rey, Heiko Neuberger, and Yves Poitevin. The European ITER test blanket modules: Progress in development of fabrication technologies towards standardization. Fusion Engineering and Design, 109–111, 2016.DOI

Paper details

vlnc:W_h82zqv3n
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents