Glass-Box Deep Reinforcement Learning for High-Speed Flow Control

PERSPECTIVE

Key points

• DRL can discover effective flow-control strategies through interaction, but its performance is bounded by the fidelity of the environment in which it learns.

• High-speed flows, with shocks, shock–boundary-layer interaction, transition and thermal loads, make training environments costly and sensitive to numerical choices.

• Numerical artefacts can become exploitable features of the control problem, making simulation fidelity a prerequisite for credible technology readiness.

• Model-based learning is data-efficient in simple flows, but its advantage can diminish in turbulent, high-dimensional and shock-dominated regimes where model errors accumulate.

• Generalization across Mach and Reynolds numbers, geometries and sensing conditions, together with robustness to noise, latency and actuator limits, remains a major barrier to deployment.

• Explainable and interpretable policies are needed to build trust, reveal the flow physics being exploited and support certification.

• Physics-aware, hybrid and foundation-model approaches could provide more generalizable and constraint-aware controllers.

• Higher technology readiness requires real-time operation, hardware compatibility, experimental validation and safety beyond nominal simulation conditions, especially at high speed.

Introduction

Controlling a fluid flow is among the oldest ambitions of fluid mechanics, and among the most consequential for engineering. Delaying separation on a wing, suppressing vortex shedding behind a bluff body or damping unsteady shock motion can cut drag, noise and structural loads. Yet the very properties that make flows worth controlling also make them hard to control: strong nonlinearity, a wide range of coupled spatial and temporal scales, sensitivity to small perturbations and, in practice, access to only a few noisy measurements. Classical active flow control (AFC) responds with open-loop forcing tuned empirically, or with feedback laws derived from linearized or reduced-order models[1, 2]. These approaches work well when the dynamics are dominated by a handful of coherent structures around a known base state, and degrade away from it.

Deep reinforcement learning (DRL) offers a different bargain. Instead of modelling the flow and then designing a controller, a DRL agent learns a control policy by trial and error, guided only by a reward that encodes the objective (Box 1). Early studies showed that such agents can suppress vortex shedding behind a cylinder or outperform hand-designed strategies[3, 4], and the approach has since been extended to a growing range of configurations[5, 6]. The appeal is clear: no explicit model is required, the objective can be non-differentiable or only sparsely available, and the learned law is free to exploit nonlinear mechanisms that linear analysis cannot reach.

The same appeal extends to high-speed flows, where shock waves, thermodynamic effects and fluid–structure interaction (FSI) make accurate reduced-order models especially difficult to build. In the transonic regime, DRL agents have been used to suppress buffet and buffeting in coupled FSI systems without an explicit dynamic model[7], to control airfoil flutter with synthetic jets and extend the flutter boundary beyond the training conditions[8], and to reduce drag on an RAE2822 airfoil, with and without lift penalties, using a high-order discontinuous Galerkin solver as the learning environment[9]. These efforts contrast with model-based strategies such as closed-loop shock-control bumps driven by lift feedback[10] and adaptive control built on time-variant reduced-order models[11], which depend on the quality of the underlying model. In the hypersonic regime, the emphasis shifts from aerodynamic efficiency to propulsion stability. Here DRL has been applied to microjet control of shock-wave/laminar-boundary-layer interaction on a compression ramp, using a recurrent surrogate of the flow to avoid the prohibitive cost of hypersonic CFD during training[12]. Higher-order schemes are employed to control hypersonic unstart within the isolator under uncertainty in back pressure[13]. It has also been combined with active disturbance rejection control to regulate scramjet fuel pulse injection, meeting thrust demands while preventing unstart[14], with the agent trained by the twin delayed deep deterministic policy gradient (TD3) algorithm[15]. Closely related work on sensing shows how limited the available information can be: deep learning can reconstruct near-wall pressure fluctuations in shock–boundary layer interaction from sparse, under-sampled data[16], and unstart in combined-cycle inlets can be predicted from only two to four surface pressure sensors[17]. High-speed flows are therefore a demanding test bed for DRL: simulations are costly, sensing is sparse and failure, such as unstart or flutter, is expensive.

Most reported successes in DRL-based AFC, however, come from low-Reynolds-number, two-dimensional simulations with idealized sensing and actuation, and even the high-speed studies above are largely numerical. It is far less clear whether the resulting policies are robust, whether they transfer to new conditions, and whether they can be trusted, let alone understood, in a wind tunnel or in flight. In this perspective, we examine these questions in turn. We argue that these challenges are manifestations of the broader sim-to-real gap: policies that perform well in controlled numerical environments may fail when confronted with the complexity, variability, and uncertainty of real fluid systems. The gap is especially wide for compressible and hypersonic flows, where high-fidelity training environments are expensive, surrogate models introduce their own errors and experimental validation is difficult. Closing this gap will require more than improved learning algorithms; it will require methods that account for the differences between simulated and physical environments and can maintain reliable performance under those differences. How effectively this sim-to-real gap is addressed will determine whether DRL can mature into a practical flow-control paradigm or remain largely a computational curiosity.

Box 1 | Deep reinforcement learning for flow control in brief

In DRL, an agent observes the state of an environment through a set of measurements $s_t$ (for example wall-pressure or shear-stress probes), applies an action $a_t$ (jet blowing, surface motion or plasma actuation) and receives a scalar reward $r_t$ (for example reduced drag penalized by actuation power). A neural-network policy $\pi_\theta(a_t \mid s_t)$ is trained to maximize the expected discounted return

where $\gamma\in[0,1)$ is a discount factor. In AFC the environment is usually a CFD solver and, more rarely, an experiment (Figure 1).

Model-free methods learn the policy directly from interaction data. Model-based methods additionally learn, or are given, a model of the transition dynamics and use it for planning or for generating synthetic experience. On-policy algorithms learn only from data generated by the current policy, whereas off-policy algorithms reuse past experience, which usually improves sample efficiency.

The simulator is the environment

The performance and reliability of DRL for AFC in high-speed flows are set, to a remarkable degree, by the compressible solver used for training. The solver defines the state space, governs the transition dynamics and determines the reward. Any systematic numerical or modelling error is therefore embedded in the learned policy. Unlike conventional optimization, DRL can amplify such errors: if a consistent numerical artefact yields an apparent gain in reward, the agent will find and exploit it¹, producing control strategies that are non-physical or that fail outside the training environment.

In high-speed flows this coupling is especially strong, because the quantities that matter for control are precisely those that compressible solvers capture least reliably. Shock-capturing schemes rely on upwinding, limiters or nonlinear reconstruction, which introduce dissipation that is both solution-dependent and nonlinear[19, 20]. Shock position, shock/boundary-layer interaction (SBLI) unsteadiness and separation-bubble dynamics are sensitive to the flux scheme, the limiter and the turbulence closure[21, 22]. At hypersonic speeds, wall heat flux and boundary-layer transition depend further on grid alignment, thermochemical modeling and transition prediction[23, 24]. Because DRL optimizes cumulative reward over long horizons, small errors in the predicted response to actuation accumulate over many control steps, and the agent may end up stabilizing numerical features, such as a limiter-switching pattern or a smeared shock, rather than true flow structures. Solver consistency and physical fidelity are therefore more critical for DRL than for most closed-loop or adjoint-based methods. Moreover, the cost of high-fidelity high-speed simulation (scale-resolving turbulence, finite-rate chemistry) limits the number of environment interactions available for training[25]. The consequences differ across speed regimes (Table 1 and Figure 2).

Regime	What the solver must capture	How errors can mislead the agent	Key references
Turbulence and transition (all regimes)	Compressible near-wall shear, coherent structures, separation; closure choice (RANS, URANS, hybrid RANS–LES, wall-modelled LES); transition onset in supersonic and hypersonic boundary layers	Agent controls closure-model artefacts; excess dissipation suppresses fluctuations and inflates apparent control authority; RANS failure in separated and shock-interacting flow; policies fragile at higher Reynolds number or finer grids	[26, 27, 28, 29, 30]
Subsonic / low Mach	Separation, reattachment, unsteady shear layers and vortex shedding; correct low-Mach behaviour of compressible solvers	Compressible schemes at low Mach number are over-dissipative unless preconditioned or low-dissipation; numerical diffusion damps shedding, so actuation effectiveness is overestimated; 2D-trained policies may exploit coherent shedding absent in 3D	[3, 31, 32, 33]
Transonic	Shock–boundary-layer interaction, shock position and motion, buffet onset and low-frequency shock oscillation	Buffet onset and shock location depend strongly on closure and scheme; spurious shock oscillations or smeared shocks are learned and exploited; poor transfer to higher fidelity or experiment	[21, 34, 35, 36]
Supersonic	Shock strength and topology, shock/shock and SBLI separation bubbles, expansion fans, entropy production	Limiter switching, Gibbs-type oscillations and carbuncle-type instabilities create an irregular, discontinuous reward landscape; agent exploits solver instabilities; SBLI response to actuators (e.g. vortex generators) is closure-sensitive	[20, 22, 37, 38, 39]
Hypersonic	Viscous–inviscid interaction, high-temperature real-gas effects, thermochemical nonequilibrium, wall heat flux, boundary-layer transition and its control (e.g. porous coatings)	Artefacts such as artificial heating (flux-scheme- and grid-alignment-dependent) dominate the reward; transition-location errors mislead stability-based control; high cost limits interactions, favouring reduced-order or multi-fidelity training	[13, 24, 40, 41, 42, 43, 44, 45]

DRL can discover effective flow-control strategies through interaction, but its performance is bounded by the fidelity of the environment in which it learns. Two solver properties matter most: what the solver lets the agent observe, set by the turbulence closure (URANS, hybrid RANS–LES or wall-modeled LES), and how much it lets the agent explore, limited by computational cost. Errors in near-wall modeling propagate directly into wall shear stress, pressure fluctuations and shock motion, the very quantities that most control objectives target[26, 27, 28, 29]. High-order, low-dissipation numerics and adequate temporal resolution are essential, because excess dissipation suppresses fluctuations, inflates apparent control authority and yields policies that are fragile at higher Reynolds number, on refined grids or in experiment. High-speed flows, with shocks, shock–boundary-layer interaction, low-frequency shock motion, transition and thermal loads, raise reward variance and make training costly and numerically sensitive[36], and nu-

merical artefacts can become exploitable features of the control problem. Scale-resolving and thermochemically complete simulations restrict the number of environment interactions, pushing practitioners towards reduced-order surrogates and multi-fidelity training[25, 45]. These improve data efficiency, but their advantage can diminish in turbulent, shock-dominated regimes where model errors accumulate. Generalization across Mach and Reynolds numbers, geometries and sensing conditions, together with robustness to noise, latency and actuator limits, remains a major barrier to deployment, and explainable, interpretable policies are needed to build trust and support certification. Physics-aware, hybrid and foundation-model approaches[46], along with machine-learning-accelerated solvers[47], could provide more generalizable controllers, but remain largely untested for shock-dominated flows. High-fidelity simulation can act as a virtual laboratory for assessing actuators, sensing and control frequency, yet higher technology readiness ultimately requires real-time operation, hardware compatibility, experimental validation and safety beyond nominal simulation conditions.

Model-free and model-based learning

Where model-based DRL works: Model-based DRL promises better data efficiency and a degree of interpretability by learning, or exploiting, a model of the flow and optimizing actions over an extended horizon. In AFC it has been implemented with learned surrogates, reduced-order representations and, in some cases, numerical PDE models[12, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60]. In low-dimensional or weakly nonlinear settings, such as laminar separation control or simplified wake stabilization, it is competitive and can require orders of magnitude fewer interactions than model-free learning[53]. This matters most where each evaluation is costly, as in high-fidelity simulation and experiment, and high-speed flows are the clearest case. Shock-resolving compressible solvers are expensive, and hypersonic simulations are prohibitive for the many episodes that model-free training requires. The most direct high-speed example is the control of hypersonic shock-wave/laminar-boundary-layer interaction on a compression ramp with microjets, where a Long Short-Term Memory network trained on CFD data served as the surrogate environment, which reduced training time and allowed the agent to learn quickly[12]. The CFD data for that surrogate still came from a second-order finite-volume solver, so the surrogate inherits the fidelity of the solver that generated it. By contrast, the model-free alternatives reported so far[4, 6] typically pay for their generality with large numbers of expensive solver calls. High-speed model-free agents have been trained directly on the flow solver, including PPO agents for transonic flutter[8], a model-free agent for buffet and buffeting in coupled FSI systems[7], unstart control for hypersonic flows[13], and a DRL agent on a fifth-order discontinuous Galerkin environment[9]. These show that model-free training is feasible at transonic conditions, but the cost scales with solver fidelity.

Model-based ideas also appear in high-speed control outside DRL. Time-variant reduced-order models have been used for adaptive buffet control[11], and classical closed-loop laws with explicit gains and delays have been used with shock-control bumps[10]. These controllers are built on a model of the flow, so they are natural baselines and natural scaffolds for model-based DRL.

Where it breaks down: The advantage fades as flow complexity grows. Fully turbulent flows involve strong nonlinearity and interactions across many scales; learning an accurate predictive model of such a system is hard, and small model errors compound over multi-step rollouts, degrading planning in high-Reynolds-number and three-dimensional flows[61, 62]. High-speed flows add difficulties of their own. Shocks are discontinuities whose position and strength change with small shifts in Mach number or angle of attack, shock–boundary layer interaction couples unsteady separation to shock motion, and in FSI problems the structure feeds back on the flow. Smooth learned models may represent such features poorly, and a surrogate fitted to a limited range of conditions may be unreliable outside it. The high dimensionality of the state compounds the problem. Practical implementations therefore compress the flow with proper orthogonal decomposition or autoencoder-based latent representations[2, 63], at the risk of discarding small-scale features that matter for control and so limiting robustness to unseen conditions. In compressible flow the discarded content may be the shock itself, since shocks produce broadband content that low-order bases capture poorly.

Model-based methods are also sensitive to uncertainty from turbulence modelling, discretization, sensor noise and incomplete state information. Realistic control typically relies on sparse measurements such as wall pressure or shear stress, which makes the problem partially observable and violates the Markov assumption underlying most RL algorithms[64]. Sparse sensing is the norm at high speed. Deep learning can reconstruct near-wall pressure fluctuations in shock–boundary layer interaction from sparse, under-sampled data[16], and inlet unstart can be predicted from only two to four surface pressure sensors in under 2 ms[17], which suggests that learned observers can supply the state information that a model-based agent lacks. Recurrent networks such as the LSTM surrogate above[12] also provide memory that partly compensates for partial observability. When the learned model departs from the true dynamics, particularly under changing operating conditions, policies can become suboptimal or unstable, and tend to be less robust than model-free or hybrid alternatives outside their training distribution[65]. This matters where failure is costly, such as flutter or inlet unstart. Ongoing efforts address this through uncertainty-aware ensembles[62], physics-informed learning that embeds conservation laws[66] and hybrid model-based and model-free schemes. A high-speed example of a hybrid is the combination of active disturbance rejection control with a TD3 agent for scramjet fuel pulse injection[14], where the classical loop handles disturbances and the learned part adjusts injection parameters to meet thrust demands and avoid unstart. Model-based DRL is unlikely to displace model-free learning in highly turbulent regimes, but it remains a valuable component of data-efficient, interpretable control. For high-speed flows, a practical division may be a model-free agent trained on a high-order solver for validation, with a surrogate or reduced-order model used for cheap pretraining and a classical controller as supervisor.

Foundation models as a way forward? Scientific foundation models (SciFMs)[46] offer a different route. Rather than learning task-specific dynamics for a single regime, they aim to capture transferable representations of fluid behaviour from large, heterogeneous datasets spanning configurations, Reynolds/Mach numbers and actuation strategies. Used less as explicit predictors for planning than as sources of latent representations or physical priors, they could reduce sensitivity to local modeling errors, improve robustness under distribution shift and allow rapid adaptation to new objectives with little extra data. For high-speed flows, a model pretrained across Mach numbers, airfoils and ramp or isolator geometries could in principle cut the cost of each new configuration, which is attractive where a single high-order or hypersonic simulation is already expensive. Their central limitation, however, is numerical fidelity: current deep-learning-based SciFMs do not deliver the high order of accuracy of traditional solvers. Unlike high-order schemes, whose error decreases at a known rate under grid and time-step refinement, learned models have no formal order of accuracy and no guarantee of convergence. Their errors are typically set by training data and architecture rather than by discretization, and they tend to damp small scales and high-frequency content. These are precisely the near-wall structures, pressure fluctuations and low-frequency shock motion on which control objectives depend. For this reason, SciFMs cannot yet replace high-order solvers as the environment in which policies are trained and certified, and are better suited to supplying priors or accelerating them, with the solver acting as the reference for accuracy. Shocks sharpen the concern, since representations learned from largely smooth data may smear discontinuities or misplace them, and the resulting errors can be exploited by an agent in the same way as numerical artefacts. SciFMs also bring other difficulties: the cost of offline pretraining and online fine-tuning, the reduced interpretability of learned representations and open questions about stability when embedded in a closed loop. They may therefore complement model-based control and high-order simulation by shifting emphasis from precise local prediction to transferable physical understanding, though this remains a nascent direction.

Choosing an algorithm

Algorithm choice shapes both learning efficiency and control robustness in flows that are nonlinear, high-dimensional and often chaotic (Table 2). Model-free methods used in flow control fall into two broad families, on-policy and off-policy, and the distinction matters because it sets how much data each interaction with the flow is worth.

Algorithm	Class	Strengths	Limitations	Flow-control examples
PPO[67]	On-policy, policy gradient	Stable training through clipped updates; easy to implement; dependable baseline	Low sample efficiency, costly with expensive simulations or experiments	[3, 68, 69]
DQN[70]	Off-policy, value-based	Reuses experience through a replay buffer; simple for discrete actuation (e.g. on/off jets)	Restricted to discrete actions; overestimates action values	[71]
DDPG[72]	Off-policy, deterministic actor–critic	Continuous actions; improved sample efficiency	Sensitive to hyperparameters; brittle under turbulent or noisy conditions	[4, 73]
TD3[15]	Off-policy, deterministic actor–critic	Twin critics and delayed updates curb overestimation; more stable than DDPG	Still deterministic, so exploration relies on added noise	[14]
SAC[74]	Off-policy, entropy-regularized	Sustained exploration; smoother control laws; balance of efficiency and robustness; suited to multistable flows	Comparative evidence on turbulent benchmarks still to be established	[13]

On-policy policy-gradient methods, notably proximal policy optimization (PPO)[67], are the most widely used for their stability and ease of implementation. The clipped objective limits the size of each policy update, which reduces the chance of catastrophic degradation, an advantage in closed-loop control where an unstable policy can excite undesirable flow modes. PPO underlies the early demonstrations of vortex-shedding suppression behind a cylinder[3], actuator-selection strategies[68] and multi-agent control of three-dimensional cylinder flows[69]. The price is low sample efficiency, because each batch of data is discarded after a few updates, which becomes a bottleneck when simulations or experiments are expensive[64].

Off-policy algorithms reuse past experience and thereby improve sample efficiency, which is valuable in flow control because collecting new trajectories can require long integration times to obtain statistically converged responses. The deep Q-network (DQN)[70] was the first of this family applied to flow control, for example in closed-loop separation control over an airfoil[71], but it is limited to discrete actions and therefore to coarse actuation. Deep deterministic policy gradient (DDPG)[72] extends off-policy learning to continuous actuation and has been used for bluff-body control in experiments and simulations[4] and for drag reduction in turbulent channel flow[73]. However, it couples a deterministic policy with strong sensitivity to hyperparameters, which can produce brittle learning dynamics and reduced robustness in turbulent or noisy conditions[74]. Twin delayed DDPG (TD3)[15] mitigates the overestimation of action values that destabilizes DDPG by using two critics and delayed policy updates; it has been adopted, for example, for scramjet fuel injection control[14].

Soft actor–critic (SAC)[74] maximizes both expected return and policy entropy, which sustains exploration and mitigates premature convergence. This is particularly relevant in multistable flows, where several attractors can coexist and greedy exploitation may trap the controller in a locally good but globally inefficient state. Entropy regularization also tends to give smoother control laws that are less sensitive to measurement noise, and off-policy learning makes efficient use of the sparse, delayed rewards typical of drag- or lift-based objectives. These properties suit high-dimensional, partially observable flows, and SAC currently appears to offer the most favourable balance of efficiency and robustness, whereas PPO remains a dependable baseline. Systematic, like-for-like comparisons on turbulent benchmarks are still needed to confirm this ranking as the field moves from low-dimensional test cases to realistic, high-Reynolds-number problems.

Rewards shape what is learned

Rewards shape what is learned

In DRL the reward is the only feedback, so its design controls the speed of convergence, the robustness of the policy and the physical relevance of the strategy that emerges [64, 75]. Naive rewards, such as instantaneous drag or lift, can be sparse or noisy and lead to unstable updates. Shaping the reward with denser signals, penalties on actuation energy or multi-objective formulations can accelerate learning and discourage non-physical shortcuts [3, 76]. Shaping must be applied with care, because a poorly chosen term can bias the agent towards suboptimal behaviour or hide the essential dynamics [77].

Learned flow-control policies are often highly sensitive to such choices: modest changes in the weights on drag, energy cost or vorticity-related metrics can produce qualitatively different actuation [3]. Domain knowledge is therefore not optional. Physics-informed rewards that encode symmetries, invariants or conservation laws steer the agent towards strategies that are effective and physically interpretable [5, 66, 78]. Penalizing violations of mass or momentum conservation, for example, can prevent exploitation of unrealistic flow manipulations, and emphasizing the suppression of coherent structures can focus control on dynamically relevant features. More broadly, the view that a well-specified reward can in principle give rise to sophisticated behaviour [75] suggests that rewards should capture cumulative flow performance, balance competing objectives such as drag reduction against actuation cost, and include regularization that prevents oscillatory actuation. In practice, reward design is an iterative process that combines domain expertise, empirical testing and guidance from RL theory.

Sensing and sampling

Sparse sensing: Practical AFC is constrained by sparse, imperfect sensing. DRL controllers can nevertheless reach effective performance with a small number of wall-mounted sensors, provided these capture the dominant dynamics such as separation or vortex shedding, and the loss relative to full-state feedback is often modest. The minimum sensing requirement is problem-dependent and tied to the dimensionality of the underlying dynamics. Flows dominated by a few coherent structures may need only a handful of well-placed probes, whereas broadband turbulence and multi-scale interactions demand richer sensing. Placement in regions of high flow sensitivity matters more than sensor count, and placement can itself be optimized within the DRL framework[79].

Noise, delay and partial observability are the main threats. Noise can bury weak but dynamically important signals, delays make the controller act on outdated information and may destabilize the loop, and policies trained under idealized sensing often lose robustness when exposed to realistic noise and latency. Recurrent architectures and temporal state augmentation can partly restore the Markov property by encoding flow history, and robust training together with observer-based hybrids will be important for deployment.

How often to sample: The sampling rate must resolve the dominant time scales of the flow, such as vortex shedding, shear-layer oscillations or energy-containing eddies[79, 80]. A practical analogue of the Nyquist criterion requires

where $f_{\max}$ is the highest relevant flow frequency. Undersampling causes aliasing, masks the causal link between state and action, and can yield policies that act out of phase with the flow or excite unintended dynamics[81]. The flow does not alone set the optimum, however, because actuator bandwidth $f_{\mathrm{act}}$ limits how fast the system can respond, and additional temporal resolution beyond it is wasted. A physically informed choice aligns both:

Spatial sensing interacts with this choice: sparse sensors may call for higher temporal rates to preserve observability, whereas dense networks can tolerate lower rates when spatial information is rich enough to reconstruct the flow[82]. Oversampling adds little and carries costs, including more data, longer episode horizons that slow training and redundant correlations that complicate function approximation[64]. Event-triggered and adaptive sampling have been proposed to balance resolution and efficiency[83].

Explainable DRL

A DRL policy is a high-dimensional function fitted to a reward, and high reward alone is a weak basis for trust. The need for more is sharpest in high-speed flows, where failure is costly: flutter can damage a structure, and unstart can cut thrust in a scramjet. A controller that is to be flown, certified or built upon must be explained: which measurements does it use, what does it do with them, and which flow mechanism does that action exploit? Explainability, in the sense used here, is the post hoc analysis of a trained policy that is treated as a black box (Box 2). It asks for an account of behaviour that already exists, without changing the policy itself. Figure 3 sketches the resulting glass-box view, in which attribution, embedding and intervention tools open the policy to inspection and tie its behaviour to flow physics.

Box 2 | Explainable, interpretable and generalizable DRL

Explainable policies are analysed after training to account for their behaviour, using attribution, embeddings or interventions. The policy itself is unchanged and may remain a black box.

Interpretable policies are transparent by construction. Their form is restricted so that inputs, structure and parameters have direct physical meaning.

Generalizable policies retain performance when conditions differ from those seen in training, such as Reynolds or Mach number, geometry, shock position, sensing and actuation layout, or the move from simulation to experiment.

The three properties are linked. An explanation can reveal which features a policy relies on, an interpretable form can force reliance on features that are physically invariant, and invariant features are the ones most likely to transfer.

Attribution and sensitivity: The most direct tools quantify how much each input drives each decision. Saliency maps and feature-importance scores, developed for visual RL agents[84], translate naturally to AFC, where inputs are sensor signals or flow-field snapshots and the question becomes which probes, which regions of the field and which time lags the policy relies on. In high-speed flows the inputs are typically a few wall-pressure or force signals, because full-field data are rarely available in experiments. Sparse-sensing studies show that a handful of surface pressure sensors can suffice to reconstruct shock–boundary layer interaction loads[16] or to predict inlet unstart[17], and DRL controllers with sparse surface pressure feedback have been developed for incompressible bluff-body flow[55]. Attribution then indicates which of those few sensors matter and at what delay. It is most informative when compared with flow physics. If the highest-attribution sensors sit near the shock foot, in the separation bubble or on the surface swept by the oscillating shock, the policy is attending to the instability it is meant to control, as in transonic buffet[7] or shock–boundary layer interaction[12]. If they sit where the flow carries no relevant information, the policy may be exploiting an artefact of the training environment.

Embeddings of policy behaviour: Dimensionality reduction of the state–action manifold, using principal component analysis or t-distributed stochastic neighbour embedding[85], can reveal whether a policy occupies a small number of distinct actuation modes, switches between them, or varies continuously with flow phase. Coloured by flow quantities such as shock position, buffet phase, pitch and plunge phase in flutter[8], separation-bubble size or thrust, these embeddings link regions of policy space to coherent structures such as shocks, vortices and shear layers, which gives an interpretable vocabulary for a policy that is otherwise numeric. Modal decompositions of controlled and uncontrolled flows[2] complement this view from the flow side, showing which structures the controller amplifies or suppresses. In early cylinder-wake studies, for example, learned actuation correlated with phase-locked forcing of vortex shedding, which points to implicit exploitation of the underlying instability[3], although the policy itself revealed little about how it did so without such targeted analysis. The analogous question for shocked flows is whether an agent locks its actuation to the shock oscillation or to a structural frequency, and whether this is the mechanism that extends a flutter boundary beyond the training conditions[8].

Intervention and causal analysis: Correlation is not mechanism. Because the DRL loop can be simulated, explanation can be interventional: individual sensor channels can be masked or perturbed, actuation can be clamped or time-shifted, and the response of both policy and flow observed. This separates sensed features that cause an action from those that merely co-vary with it, and actions that change the flow from those that do not[86]. In a closed loop the distinction is subtle, because the policy changes the flow that it then senses, so interventions must be designed with the feedback in mind. High-speed flows add specific tests. Shifting the actuation phase relative to the shock oscillation, silencing the sensor nearest the shock, or changing the Mach number or angle of attack slightly shows whether the policy tracks the shock or only a correlated signal. These interventions also allow a direct comparison with classical controllers such as lift-feedback shock-control bumps[10] or reduced-order-model-based control[11], whose gains and delays are explicit, and with actuation concepts such as boundary-layer suction[87] or electrical discharge[88] that act on the shock structure directly. If a learned policy reproduces the logic of such a controller, that is evidence of physical grounding. If it departs from it, the departure is either a discovery or a warning.

Explanations are models of a policy, not the policy, and can be unfaithful, so faithfulness should be tested: removing sensors marked important should degrade control, and removing those marked irrelevant should not. Explanation also audits for numerical artefacts that agents exploit. In compressible flows, an agent trained on a low-order solver may be tuned to shock smearing rather than shock dynamics, whereas high-order solvers resolve shock oscillations more sharply[9, 13]. Learned surrogates, which can cut hypersonic training cost[12], pose the same risk, since explanations may describe the surrogate’s errors rather than the flow. Re-running attribution and intervention tests on the full-order solver, and ultimately on experiments, exposes such failures before a policy is trusted for tasks such as unstart prevention[14].

Interpretable DRL

Where explainability analyses a finished policy, interpretability builds transparency in. An interpretable policy is one whose structure can be read directly: its inputs, functional form and parameters carry physical meaning, so that its behaviour follows from its design rather than from an after-the-fact account (Box 2). This is the stronger property and the harder one to obtain, because it constrains the very flexibility that makes DRL attractive. The constraint is worth accepting in high-speed flows, where the controller must hold up in regimes that cannot be flight-tested cheaply, and where a policy whose logic is open to inspection is far easier to certify than one that is not.

Controlling the policy class: The simplest route is to restrict what the policy can express. Small networks with few inputs, bounded actuation and smoothness regularization yield control laws that can be inspected and bounded, and they reduce sensitivity to measurement noise. This suits high-speed problems, where sensing is naturally sparse, such as a few surface pressure probes[16, 17], and actuation is naturally bounded, such as synthetic jets[8] or microjets[12]. Structured policies go further by building known invariants or symmetries into the architecture, so that the admissible laws are consistent with the physics from the start[66]. Examples include non-dimensional inputs that make a policy portable across Mach number, or actuation constrained to respect the phase relation between pitch and plunge. This is attractive wherever an unconstrained network would have to rediscover such structure from data.

Kolmogorov–Arnold networks: Interpretability can also be sought in the network architecture itself. Kolmogorov–Arnold networks (KANs)[89] replace the fixed activations and linear weights of a multilayer perceptron with learnable univariate functions on the edges, so that each connection can be plotted, pruned and, where the learned shape is simple, replaced by a closed-form expression. A KAN policy, or a KAN critic, can therefore be read as a composition of one-dimensional functions that relate each sensed quantity to the actuation, which makes it a natural bridge between neural and symbolic control laws. For high-speed flows this means a policy that maps wall-pressure signals to jet amplitude through curves that can be compared directly with the gain and delay of a classical shock-control law[10]. KANs have been compared with multilayer perceptrons on scientific problems such as differential equations and operator learning[90], but their use as DRL policies for compressible flow has barely been explored. Open questions include their sample efficiency and cost inside a costly CFD loop, their behaviour under noisy sensing, and whether the learned edge functions stay simple enough to interpret when the dynamics involve discontinuities such as shocks.

Learning in physical coordinates: A further route changes the space in which the policy acts. Coupling DRL to reduced-order models built from proper orthogonal decomposition (POD), dynamic mode decomposition (DMD) or resolvent analysis lets the agent act on a handful of modal amplitudes instead of raw sensor data, and projects its strategy onto physically meaningful bases[2, 59]. This route is poorly suited to high-speed flows in which shocks appear. POD and DMD represent the flow as a superposition of global, smooth modes, whereas a shock is a near-discontinuity whose spectral content extends to ever higher wavenumbers, tending to infinity in the inviscid limit. A low-order basis therefore converges slowly, needs many modes to resolve even a single shock, and tends to smear the shock or produce spurious oscillations around it. The difficulty grows when the shock moves, because a translating discontinuity is not well described by a linear combination of fixed modes, so the truncated model loses the very feature that controls loads, separation and unstart. In high-speed flows the natural coordinates are well known: the shock position and the buffet mode in transonic flow, pitch and plunge in flutter, the leading-edge position of the shock train in an isolator[87, 88], and the back-pressure ratio that governs inlet unstart[13, 17]. The state is then a set of named flow structures, and the policy is a map between them that can be examined directly. Time-variant reduced-order models have already been used for adaptive buffet control[11], and pairing such models with a DRL agent is a natural next step, provided the shock is tracked explicitly through such coordinates rather than left to a modal expansion. The price is that anything outside the retained modes is invisible to the agent, which matters when shocks produce broadband content that low-order bases capture poorly.

Distillation and hybrid structure: A fourth route compresses a trained policy into a compact, human-readable form, such as a low-order symbolic expression or a shallow decision structure, and verifies that the compact law retains most of the performance. Programmatically interpretable policies[91], decision-tree extraction[92] and sparse regression of governing or control laws[93] provide the tools, and a distilled law for buffet or flutter suppression would be directly comparable with the closed-loop laws used in classical practice[10]. Hybrid designs reach a similar goal from the other direction, using a classical controller such as a linear quadratic regulator or model predictive control as the scaffold and letting DRL learn only a bounded correction[1]. The combination of active disturbance rejection control with a TD3 agent for scramjet fuel injection[14] is an example from the high-speed literature: the classical part is interpretable and certifiable, and the learned part is confined to where it adds value. We return to such hybrids below.

Rewards and constraints as interpretable specifications: The objective itself can be made interpretable. Physics-informed rewards that penalize actuation energy, violation of conservation laws or non-physical manipulation[5, 78] state what the controller is meant to do in terms that engineers can audit. In high-speed settings the specification is often explicit already: drag reduction with or without a lift penalty[9], thrust demands together with unstart avoidance[13, 14], or bounds on pitching and plunging amplitude[8]. Constrained or safe RL formulations make actuator limits, energy budgets and safety margins explicit rather than implicit in reward weights, so that a hard boundary such as the unstart limit is enforced by construction instead of being traded against thrust.

Each of these routes gives something up. A restricted policy class cannot represent every nonlinear strategy that an unconstrained network can, and the very behaviours that make DRL interesting, such as exploitation of subtle instabilities or cross-scale interactions, may be the ones a constrained form cannot express. Compressible flows sharpen the trade-off, because shock dynamics are strongly nonlinear and a policy that is too simple may miss the nonlinear shock–boundary layer coupling it is meant to exploit. We therefore expect a Pareto frontier rather than a single answer, and the field would benefit from reporting both control performance and a measure of policy complexity. The payoff is more than trust. An interpretable policy turns learned behaviour into a hypothesis that conventional analysis can test, which is how DRL can contribute to flow physics rather than only to flow engineering.

Generalizable DRL

Generalization is where the promise of DRL is most exposed, and high-speed flows expose it further. Policies are trained at a fixed Reynolds number and Mach number, on a fixed geometry, with a fixed sensor and actuator layout, in an idealized solver, and are then expected to work elsewhere. In practice, zero-shot transfer is limited. A policy exploits the coherent structures, dominant frequencies and instability mechanisms of its training regime, and even moderate changes in operating conditions alter the spectrum and the balance of the governing mechanisms enough to degrade control rapidly. Controllers trained to suppress vortex shedding are tuned to a narrow band of Strouhal numbers and wake dynamics[3, 4]. In compressible flow the dependence is at least as strong, because small changes in Mach number or angle of attack move the shock, shift the buffet onset boundary or change the flutter margin, and the mapping from state to action changes with them. Standard agents thus appear to learn environment-specific strategies rather than physics-invariant laws. The few high-speed studies that report generalization are therefore notable. A DRL flutter controller has been reported to reduce pitching and plunging oscillations across a range of Mach numbers and to extend the flutter boundary beyond the training conditions[8], and a machine-learning unstart predictor has been designed for a wide speed range with only a few surface pressure sensors[13, 17]. Such results show that transfer is possible, but each covers one axis over a stated range, and none shows that the policy has learned an invariant law.

Axes of generalization: It helps to separate what is being generalized over. Parametric generalization concerns Reynolds number, Mach number, angle of attack and operating conditions. For a transonic airfoil this includes the lift-penalty weight in the reward, since drag reduction with and without a lift penalty are different tasks[9]. Geometric generalization concerns body shape, boundary conditions and the location of actuators and sensors, for example a different airfoil section, a different isolator or ramp geometry, or the placement of jets and pressure probes. Regime generalization concerns the move between laminar, transitional and turbulent states, and between compressible regimes where the dominant mechanisms change, from attached transonic flow to buffet, from supersonic to hypersonic shock–boundary layer interaction, or from started to unstarted inlet operation. Finally, generalization from simulation to experiment, taken up in the next section, concerns the fidelity gap, sensor noise, latency and actuator limits. A policy can generalize well along one axis and fail along another, so claims of generalization should name the axis and the range over which it was tested.

Training for diversity: Domain randomization varies key parameters during training so that the policy must rely on features common to many flow realizations[94]. For high-speed flows the natural parameters are Mach number, angle of attack, structural stiffness and damping in FSI problems, and the shock position at the start of an episode. It does not guarantee transfer, but it enlarges the range of conditions with acceptable performance, and randomizing noise, latency and actuator response begins to address the simulation-to-experiment axis. Multi-fidelity and curriculum training begin with cheap, low-fidelity or low-Reynolds-number data and refine on expensive data, aided by faster solvers[47]. This is attractive when a hypersonic or high-order simulation is prohibitive, and learned surrogates already serve as cheap training environments for hypersonic control[12]. Their benefit is plausible but not yet established for flow control, and the central risk is importing the biases of the cheap model. In compressible flow this risk is concrete: a surrogate fitted to a limited range of conditions may be unreliable outside it, and a low-order solver may smear the shock and alter its oscillation, whereas high-order schemes resolve it more sharply[9]. A policy trained on the cheap model can thus be tuned to its errors, which is a transfer failure in its own right.

Training to adapt: Transfer learning pretrains a policy in a simpler setting and fine-tunes it in a harder one, reusing control-relevant representations such as separation detection or shear-layer manipulation, and so cuts training cost[95]. Transfer from two- to three-dimensional bluff-body flow has been demonstrated[95, 96], and the same logic suggests pretraining on two-dimensional airfoil buffet or shock–boundary layer interaction before fine-tuning on three-dimensional wings or ramps, where simulations are far more expensive. Meta-learning[97] and multi-task RL[98, 99] optimize explicitly for fast adaptation, which introduces an inductive bias towards generalization across flow and geometry parameters. Applications to fluid mechanics are at an early stage, but they point to controllers that adapt with a few interactions instead of being retrained from scratch, for example, across a family of Mach numbers or after a change in structural properties.

Learning what is invariant: The deepest route is to make policies depend on what does not change. Representations that encode physical structure, whether built in through symmetries and invariants[66], obtained from reduced-order bases[2], or pretrained as latent priors in scientific foundation models[46], should transfer further than raw-sensor policies because they discard condition-specific detail. In high-speed flows, non-dimensional inputs and physically named coordinates are the obvious candidates: shock position relative to the chord, pitch and plunge amplitudes, or the back-pressure ratio that governs unstart[13, 17]. Reduced-order models that vary in time have already supported adaptive buffet control across changing conditions[11], and a DRL agent acting on such coordinates is a natural next step. Here generalization meets interpretability: a policy that is shown, by explanation or by construction, to rely on invariant features is a better candidate for transfer than one that is not. Classical laws offer a useful benchmark, since lift-feedback shock control[10] works through explicit gains and delays that carry across conditions by design. Foundation models carry their own costs, including pretraining and fine-tuning expense and uncertain stability in closed loop, and shocks, which are discontinuous, may be hard for smooth latent representations to capture.

Progress needs shared protocols. Policies should be evaluated on held-out Reynolds numbers, Mach numbers, angles of attack, geometries and sensor layouts rather than only on the training condition, with zero-shot and few-shot performance reported separately and with robustness to noise and delay tested explicitly. For high-speed flows the test should also include the regime boundaries, such as conditions near buffet onset, the flutter boundary or the unstart limit, because these are where a failure to generalize is most costly and where a policy trained deep inside a regime is most likely to fail. Without such benchmarks, improvements in learning efficiency are easy to mistake for improvements in generalization. The measure that ultimately matters for deployment is not how well a policy performs where it was trained, but how gracefully it degrades where it was not.

From simulation to experiment

Moving from simulation to experiments or real systems is the hardest step, and in high-speed flows it is harder still. The central obstacle is the simulation-to-reality gap (Figure 1): high-fidelity simulations rely on idealized boundary conditions, perfect state observability and deterministic dynamics, assumptions that rarely hold in a laboratory[1, 100]. Policies trained in simulation may exploit numerical artefacts or control authority that cannot be realized, and become unstable or suboptimal when deployed. The DRL studies of transonic buffet, flutter and drag reduction[7, 8, 9], hypersonic shock-wave/boundary-layer interaction[12, 13] and scramjet fuel injection[14] are numerical, so none has yet crossed this gap. Experimental work on shock control does exist, such as the stabilization of Mach 2 shock trains with electrical discharges[88], but it uses fixed actuation rather than a learned policy. Zong et al. [101] ran DRL on a field-programmable gate array (FPGA), achieving 1–10 kHz control, two orders of magnitude faster than CPU-based frameworks. In Mach 2 supersonic backward-facing-step flow, the agent learned an effective control law within about ten seconds, increasing shear-layer mixing by 21.2%.

Closing the distance between the two is a central task for the field. High-speed facilities add their own constraints: run times are often short, test conditions are costly to reproduce, and a failure such as unstart or structural damage can end an experiment or destroy a model. This limits the trial-and-error learning on which DRL depends, so much of the training must still happen in simulation, and the gap matters more.

Modelling error, noise, latency and actuation uncertainty: Modelling errors change the perceived dynamics and can invalidate the state–action relationships learned in training[60]. In compressible flow the error often sits in the shock. Shock position, oscillation amplitude and frequency depend on the shock-capturing scheme, limiters, numerical dissipation and turbulence modelling, so a policy trained on a low-order solver may be tuned to the smearing of the discontinuity rather than to its physical dynamics, whereas high-order schemes resolve shock oscillations more sharply[9]. Surrogate environments add another layer, because a policy trained on a learned model of the flow inherits that model’s errors, and the CFD data behind it carry the fidelity of the solver that produced them[12]. In FSI problems the structural model is a further source of mismatch, since stiffness, damping and unmodelled nonlinearities in a real structure differ from those assumed in simulation[7, 8].

Sensor noise corrupts observations and effectively turns a fully observable decision process into a partially observable one, for which standard algorithms are not designed. High-speed experiments rarely offer more than a few wall-pressure signals, which are affected by noise, vibration and thermal effects. Sparse-sensing studies show that useful information can be extracted from such data: near-wall pressure fluctuations in shock–boundary layer interaction can be reconstructed from sparse, under-sampled measurements[16], and inlet unstart can be predicted from only two to four surface pressure sensors[17]. These results suggest that learned observers could supply the state estimate for a DRL controller, although neither study tests this. Noise-aware training, in which the same sparse, noisy signals available in the experiment are used as the agent’s observations, is a direct way to prepare for the sensing conditions of deployment.

Latency in sensing, computation and actuation introduces delays that can destabilize closed-loop control, especially when relevant time scales are short[102]. Time scales are short at high speed. Shock oscillation, buffet and inlet unstart evolve quickly, so the whole loop of sensing, inference and actuation must fit inside them, and this is our own estimate rather than a finding of the cited studies. Sub-2-ms prediction of unstart from surface pressure has been reported[13, 17], which shows that the sensing and inference side can be fast, but actuator response and hardware delays must be added. Actuation uncertainty, including limited bandwidth, saturation, hysteresis and variable response, makes it hard to reproduce the fine-grained strategies learned in simulation[100]. Synthetic jets and microjets, which several of the studies above use[8, 12], have finite bandwidth and amplitude limits, and the conditions under which a synthetic jet forms depend on its operating parameters[103], so a policy that assumes ideal jet response may ask for forcing that a real actuator cannot deliver. Electrical-discharge actuation acts on the shock structure directly[88], but its response in a given facility must be characterized before it can be modelled in the training environment. Boundary-layer suction is another candidate, studied numerically for shock trains[87].

All together these factors limit real-time performance and robustness, and motivate the domain randomization and adaptation strategies described above, together with noise-aware training, reduced-order sensing and hybrid model-based and learning-based approaches. For high-speed flows, a practical route is to randomize Mach number, structural parameters, sensor noise, latency and actuator response during training, test the policy on a higher-fidelity solver and in hardware-in-the-loop experiments, and keep a classical controller as supervisor to protect against unstart or structural overload. This sequence is our proposal, and it builds on hybrid designs such as active disturbance rejection control combined with a TD3 agent[14].

Beyond the efficiency–accuracy trade-off

AFC is often assumed to force a choice between efficiency and accuracy. High-fidelity simulation and dense sensor–actuator networks are accurate but expensive and may rule out real-time use, whereas low-dimensional models and coarse sampling are cheap but compromise the fidelity of learned policies[2, 3]. DRL weakens this dichotomy. Off-policy, entropy-regularized algorithms use experience replay and structured exploration to achieve strong performance with sparse data or reduced-order representations[5, 74], and hybrid schemes that couple DRL with reduced-order models or physics-informed surrogates train efficiently while retaining the dynamics that matter[66, 104]. Accuracy should in any case be judged not only by instantaneous metrics such as drag reduction, but also by robustness, generalization across Reynolds numbers and interpretability. On those terms the trade-off is a product of conventional modelling constraints, not a fundamental limit.

This is the logic of the hybrid frameworks introduced above. Coupling DRL to reduced-order models gives the agent a low-dimensional latent space that retains the dominant structures, which improves sample efficiency, stabilizes learning and readily accommodates physics-informed rewards, constraints and symmetries[2, 5, 59, 66]. Using classical control as a scaffold confines exploration to a sensible neighbourhood while leaving DRL free to discover nonlinear actuation patterns[1, 3]. What makes hybrids compelling is that the same construction addresses several of the challenges reviewed here at once: it lowers the cost of interaction, supplies interpretable structure, and carries the invariants that help policies transfer across conditions. Hybrid design thus offers one answer to the efficiency, interpretability and generalization problems instead of three separate ones, although how far it can be pushed in fully turbulent, high-speed flows is still open.

Outlook

DRL has shown that learned controllers can match classical flow control in simulation. Turning these demonstrations into working technology is now a problem of validation more than of algorithms, and it is most acute in high-speed flows. Transonic, supersonic and hypersonic systems combine shock-dominated dynamics that depend on the numerical scheme, costly high-fidelity simulation, sparse sensing, millisecond time scales and failures, such as flutter, buffet and inlet unstart, that are expensive or irreversible. In this setting a black-box policy is a liability. A controller that performs well but cannot say why will not be trusted near the flutter boundary or the unstart limit, and it cannot be certified for flight.

The way forward is glass-box AI: controllers whose behaviour can be inspected, explained and bounded. Interpretability should be built into the policy through physics-informed rewards, hybrid architectures that pair learned components with reduced-order models or classical control, and sparse, symbolic or otherwise compact representations whose terms can be tied to shock motion, boundary-layer separation or pressure recovery. Explainability should be applied after training to identify which sensors, flow features and states drive each action, to check that the policy exploits real flow physics rather than solver-specific artefacts, and to expose its failure modes. Generalizability follows from the same transparency. A policy that encodes the physical mechanism it exploits can be expected to transfer across Mach numbers, angles of attack, geometries and fidelity levels, and when it fails, the failure can be traced to a cause and not merely observed. Each property supports the others: interpretable policies are easier to explain, explainable policies are easier to test for generalization, and policies that generalize for identifiable reasons are easier to certify.

Closing the gap between simulation and flight requires this evidence at every level of technology readiness. Policies must be verified across numerical schemes and fidelity levels, designed around hard structural, thrust and unstart limits, and tested hardest near regime boundaries, then confirmed through hardware-in-the-loop experiments and blowdown or high-enthalpy facilities. Inference must fit within the time scales of the flow using the few sensors a real vehicle can carry, and a classical supervisory layer able to override the agent near the boundary offers a practical route to safety. Progress should be measured against strong classical baselines on open canonical cases, such as a transonic airfoil, a compression ramp and an isolator, with explainability and generalization reported alongside performance. Whether learned controllers are adopted in high-speed flight will depend less on how well they perform than on whether their behaviour can be understood, trusted and verified.