WorksEngineering
Reinforcement Learning Optimized Flight Control for Interception of High-Altitude Maneuvering Targets
Abstract
We implement and compare two approaches to improving flight control system (FCS) performance using reinforcement meta-learning (meta-RL). Both approaches are evaluated in a terminal phase hypersonic intercept engagement requiring hit-to-kill accuracy against a highly agile maneuvering target. In the first approach (FCS-RL), meta-RL is used to optimize a full FCS that maps acceleration tracking error and other measurements directly to commanded channel deflection rates of change, combining pitch, yaw, and roll control into a single policy. In contrast, the second approach (FCS-3L-RL) starts with a standard FCS (FCS-3L) implementing a three-loop autopilot for pitch and yaw control, but adapts the scheduled autopilot gains in real time using a policy optimized with meta-RL. Both approaches are compared to the FCS-3L in a 6-DOF simulator that captures most factors relevant to guidance and flight control. Simulations show that under ideal conditions the three systems have comparable performance. However, under conditions of aerodynamic uncertainty, radome refraction, or target evasive maneuvers, the FCS-RL significantly outperforms the FCS-3L-RL with improved intercept probability and less control effort. In turn, the FCS-3L-RL outperforms the FCS-3L by a similar margin. Importantly, we demonstrate that both FCS-RL and FCS-3L-RL generalize well to novel engagement scenarios, flight conditions, and target maneuvers that are not experienced during the optimization process.
Introduction
Flight control for control surface actuated interceptors tasked with intercepting actively evading targets in the high altitude hypersonic flight regime with hit-to-kill accuracy is a significant challenge. Consider that strike weapons fielded by a peer adversary will likely actively evade interception, requiring the interceptor to be capable of significant terminal phase maneuvering. These maneuvers can result in unsteady effects, boundary layer separation, and shock–boundary layer interaction [1, 2]. Aerodynamic modeling is further complicated by the extreme aero-thermal environment associated with hypersonic flight, where vehicle structures undergo stress caused by coupling of Fluid-Thermal-Structural Interactions (FTSI) and significant aerodynamic heating typical in hypersonic regimes [3]. This results in time-varying airframe and control surface deformation that are functions of both peak and integrated heating, which leads to vehicle aerodynamics differing substantially from that assumed during the design of the FCS [4, 5]. This aerodynamic uncertainty will require the flight control system (FCS) to adapt to differences between the simulated and deployment environments. The design of the FCS is further complicated by parasitic interaction between the guidance and flight control systems due to radome refraction [6, 7, 8]. Finally, high altitude engagements using only control surfaces actuators result in limited acceleration capability due to the low dynamic pressure.
Most currently deployed flight control systems for cruciform geometry skid-to-turn interceptors use the three-loop autopilot (TLA) [9, 10, 11] for pitch and yaw control, and a two-loop proportional-integral (roll rate and roll) controller for roll control. The TLA uses acceleration and rotational velocity measurements to track a commanded acceleration in the given channel (pitch and yaw). The FCS gains are optimized over a group of design points, typically angle of attack, sideslip angle, Mach, and altitude, and during flight the gains are interpolated between these design points to give optimal performance over the flight envelope. This approach gives both fast response and robust tracking to commanded acceleration. However, the standard TLA has limitations, including the assumptions of linear dynamics and no cross channel coupling, and simplified second order actuator model that does not account for fin limits and slew rate. Moreover, the three-loop architecture is a linear controller, which might constrain performance. Finally, as we show in the following, performance can degrade under high levels of aerodynamic uncertainty.
Consequently, considerable research has been focused on augmenting the standard three-loop autopilot, as well as exploring alternative approaches to interceptor flight control. Model-reference adaptive control (MRAC) methods [12, 13] have been used to augment conventional three-loop autopilots through supplemental control commands that drive the missile acceleration response toward that of a desired reference model, improving tracking performance under aerodynamic model uncertainty. Despite these benefits, MRAC has several limitations. Conventional MRAC stability and tracking guarantees rely on plant-model matching conditions, which may be violated under substantial aerodynamic uncertainty or changes in control effectiveness [14]. Furthermore, the reference model may demand a response that exceeds the actuator constraints, resulting in saturation and degraded tracking performance [15]. Adaptive controllers can also be sensitive to unmodeled high-frequency dynamics and measurement disturbances, potentially compromising stability, particularly at high adaptation gains [16]. Finally, asymptotic tracking guarantees do not necessarily ensure satisfactory transient performance, motivating modified adaptive architectures that explicitly address transient response [17].
More recently, reinforcement learning (RL) has been investigated both for optimizing conventional missile autopilots and for replacing their prescribed feedback architectures. Shin et al. [18] employed deep reinforcement learning to optimize the gains of a conventional three-loop autopilot, preserving its underlying control structure while allowing state-dependent gain adaptation. Taligatos et al. [19] compared RL-based optimization of conventional controller gains against a fully learned controller that directly commands control surface deflections, highlighting tradeoffs in control effort versus overshoot between the two methods. Similarly, Candeli et al. [20] employed RL to directly synthesize a missile autopilot, demonstrating its effectiveness under nonlinear aerodynamic dynamics and model uncertainty. Wang et al. [21] extended this approach to integrated guidance and control for interception of highly maneuverable targets, with an RL policy directly commanding pitch and yaw control surfaces. Gaudet et al. [22] used a hypersonic point defense intercept application to compare the effectiveness of using reinforcement meta-learning (meta-RL) to optimize a separate FCS policy as compared to optimizing an integrated guidance and control policy, and in [23] investigated meta-RL for an integrated guidance and control system in an air-to-air intercept application. Collectively, these studies illustrate the potential of RL to improve conventional FCS performance.
In the following, we explore two approaches to apply reinforcement meta-learning (meta-RL) to optimize an adaptive FCS capable of hit-to-kill (miss less than 0.25 m) terminal-phase performance against maneuvering targets over a high-altitude hypersonic flight envelope using only control surface actuation. In the first approach (FCS-RL), meta-RL is used to optimize a full FCS mapping acceleration tracking error and other measurements directly to channel deflection rates of change, combining pitch, yaw, and roll control into a single policy. In contrast, the second approach (FCS-3L-RL) starts with a standard FCS (FCS-3L) implementing a three-loop autopilot for pitch and yaw control, but adapts the scheduled autopilot gains in real time using a policy optimized with meta-RL. This work has some similarity to [19] in that we explore using RL to augment a standard FCS as well as optimizing a full meta-RL FCS. However, our work uses a much higher fidelity simulator, uses proximal policy optimization in place of deep deterministic policy gradient, includes testing over full trajectories, evaluates performance under aerodynamic uncertainty and a large flight envelope, and uses meta-RL instead of RL. The recurrent policy used in our implementation of meta-RL allows the FCS-RL to implement an integrator, which we later show is essential for the RL optimized FCS to operate over a large flight envelope.
The paper is organized as follows. § 2 describes the simulation environment and engagement scenarios, § 8 covers the development of the FCS-3L benchmark, IV.A gives background for RL and meta-RL and § 10 and § 10.A formulate the FCS optimization problems in the meta-RL framework. § 12 describes the benchmark FCS and compares the performance of the meta-RL and benchmark flight control systems. § 13 gives a brief discussion of certification issues. Conclusions are given in § 14
Simulation Environment
In this section, unless a variable has the "B" (body frame) or "S" (LOS stabilized frame) superscript, it is assumed to be in the inertial reference frame. The simulator, taken from [24], has sufficient fidelity to capture most factors relevant to flight control. Specifically, our simulator is 6-DOF, includes flight condition estimation, a guidance system, a non-linear radome model, an actuator model that accounts for actuator bandwidth, fin limits, and fin rate limits, an inertial measurement unit (IMU) that models noise, bias, and placement distance from center of gravity, a radar receiver with line-of-sight (LOS) and range dependent noise, delays between IMU and radar measurements, modeling of aerodynamic uncertainty, and explicit modeling of LOS recovery using a strap down seeker. Importantly, we optimize and test the flight control systems taking into account aerodynamic model uncertainty, which would be expected in the hypersonic flight regime.
The simulator is shown in Figure 1, where all systems run at 500 Hz. Modeling relative velocity estimation in this scenario has a significant performance impact [24]. Consequently, to better compare the effectiveness of the three flight control systems, we assume ground truth knowledge of relative velocity, and leave full relative state estimation to a separate paper [24]. However, we model flight condition estimation, including angle of attack, sideslip angle, Mach, altitude, and change in attitude from sensor measurements, as described later in § 2.C.5 and II.C.7.

Figure 1. Simulator Diagram
The interceptor G&C system must satisfy a load constraint, where the norm of the and body frame forces acting on the interceptor (see § 2.D) must be less than 75 g: , where is the magnitude of the gravitational acceleration at the Earth’s surface. The look angle constraint requires the maximum angle between the body frame x-axis and the body frame LOS vector to be less than . Rather than directly impose the look angle constraint by terminating an episode when the constraint is violated, we instead latch the G&C system observables at their value prior to the violation for as long as the constraint is violated. Consequently, it is possible to violate the constraint for short periods and maintain good performance, noting that this constraint is always violated close to the end of the engagement. Note that if we had included relative velocity estimation using a Kalman filter, the filter would propagate the state open loop (no measurement update) when the target is out of the field of view.
Engagement Scenario
Referring to Figure 21, the missile position vector, missile velocity vector, target position vector, and target velocity vector are shown as , , , . We can also define the relative position and velocity vectors and . The elevation angle is the angle between and its projection onto the x-y plane. We randomly generate the target’s initial velocity vector such that lies within a cone with axis and half apex angle . In we define a heading error () as the angle between the interceptor’s initial velocity vector and the velocity vector associated with the lead angle required to put the interceptor on a collision heading with the target. The interceptor initial conditions are listed in Table 1.

Figure 2. Engagement
| Parameters Drawn Uniformly | Min | Max |
| Elevation Angle to Target (degrees) | 0 | 10 |
| Azimuth Angle to Target (degrees) | -180 | 180 |
| Altitude (m) | 15000 | 25000 |
| Missile Velocity Magnitude (m/s) | 2300 | 2600 |
| Heading Error (degrees) | 0 | 5 |
| Initial Angle of Attack (degrees) | -5 | -5 |
| Initial Side Slip Angle (degrees) | -5 | 5 |
| Initial Range to Targeted Asset (m) | 20000 | 25000 |
| Target Velocity Magnitude (m/s) | 3000 | 4000 |
| Target Velocity Cone Half Apex Angle (degrees) | 0 | 15 |
| Interceptor Maximum Acceleration (@ 2500 m/s 20000 m, applied to each component) (g) | - | 20 |
| Target Evasion Acceleration (@ 2500 m/s 20000 m, applied to each component) (g) | 0 | 4 |
| Target Weave Period (s) | 0.5 | 3.0 |
Table 1. Simulation Initial Conditions
Target Evasive Acceleration Model
During optimization, we consider three types of evasive maneuvers, a step maneuver, a weaving maneuver, and a jinking maneuver, and add different maneuver types to test the generalization capability of the optimized policies. The weaving maneuver has frequency shared for each axis and phase independent for each axis in , and randomized phase offset and frequency. In contrast, the jinking maneuver is only randomized in frequency, shared for each axis. Each of these maneuvers has a 1/3 probability of being selected at the start of each maneuver. At each simulation step, the target’s evasive acceleration vector is modified by removing the component parallel with the target velocity vector, and then adjusted for dynamic pressure, with a nominal acceleration capability of 4 g at an altitude of 20 km and a speed of 2500 m/s. Thus, the target acceleration vector is always orthogonal to the target velocity. The maneuvers have varying acceleration levels and random start time, duration, and switching time. At the start of each episode, with probability 0.5 the maneuvers in that episode use the full evasion acceleration capability, and with probability 0.5 the full acceleration is scaled uniformly by a factor uniformly distributed between 0 and 1. Note that the target drag orthogonal to the interceptor LOS is also perceived as a maneuver; the target drag model is shown in § 2.E. Sample target maneuvers are shown in Figure 3, which show evasive acceleration only (not drag orthogonal to LOS).

Figure 3. Sample Target Maneuvers (only evasive acceleration shown)
Interceptor Components
Radome Model
We calculate the refracted azimuth and elevation angles as shown in Eqs. (1a) through (1b), where our nominal value of is 0.5 mrad, and , are set to 3.0. Note we assume that there is a combination of software (such as compensation lookup tables) and hardware (such as radome shaping) that reduce the effective radome refraction to these values.
The refraction angle is shown in Figure 4 for various values of . Note that the radome slope is given by the slope of the curves in the figure.

Figure 4. Radome Refraction Angle as Function of Look Angle
Seeker Model
We model sensor noise (elevation, azimuth, and range) as Gaussian noise, using a standard deviation of 1% of range for range dependent range noise and 0.2 mrad for angle noise. Obtaining this level of angle noise requires a dual model seeker (radar for range, infrared for angles). This improves angular resolution, target discrimination, and resistance to countermeasures. Note that our assumed noise level is post signal processing. We model the lag of the seeker signal processor as a pure delay of 4 ms.
Rate Gyro Model
Our rate gyro model corrupts the ground truth rotational velocity with both scale factor bias and Gaussian noise, as shown in Eq. (2a), where denotes an dimensional uniformly distributed random variable bounded by that is generated at the start of each episode, and each dimension of the random variable is independent. We use and . The rotational velocity is passed through a two pole low pass filter with a 100 Hz natural frequency and 0.707 damping, as shown in Eq. 2b.
Accelerometer Model
We account for accelerometer placement using Eq. (3a), with , where is the accelerometer location, which we assume to be -25 cm using the reference frame shown in Figure 5. Similarly to the rate gyro model, the body frame acceleration is corrupted with a scale factor error as shown in Eqs. (3b) and (3c), where we use , and "LPF" models the accelerometer dynamics as a two pole low pass filter with a 100 Hz natural frequency and 0.707 damping, and .

Figure 5. Interceptor Geometry
Line of Sight Recovery
Referring to Figure 1, the refracted body frame line of sight unit vector from the dual-mode receiver block is input to the LOS recovery block. This block uses the estimated change of attitude since the start of the engagement to rotate into the reference frame corresponding to the interceptor’s attitude at the start of the engagement. Specifically, at each guidance step we map to a direction cosine matrix , and compute the recovered (stabilized) LOS as .
The estimated change in attitude is parameterized as a quaternion, and is computed by integrating as shown in Equation (8), where is reset at the start of each episode to . In our simulation model, we approximate this integration using fourth-order Runge-Kutta integration. In the unrealistic case where we have access to the ground truth value of , this completely decouples from body rotation. However, since is corrupted with a scale factor error and Gaussian noise, in general , and the LOS recovery will be imperfect. The combination of imperfect LOS recovery and radome refraction result in a false indication of target motion. This results in increased miss distance, and can potentially destabilize the G&C system. This is discussed in more detail in [6, 7, 25].
Guidance System
The guidance system implements true proportional navigation (TPN) [26] to calculate a commanded acceleration from the LOS rotation rate, as shown in Eqs. (5a) through (5c), where . This commanded acceleration is then rotated into the body frame using the estimated change in attitude and adjusted using g-limits. These g-limits assume a maximum acceleration of 25 g at an altitude of 20 km and a speed of 2500 m/s, and are adjusted for dynamic pressure. This body frame commanded acceleration is then used to compute the acceleration tracking error according to , where is the accelerometer measurement and indicates we are computing the error over the body frame y and z components. We use for the navigation ratio.
Flight Condition Estimation
The FCS requires an estimate of angle of attack, , sideslip angle , altitude , and Mach . These observations are generated by integrating to obtain an estimate of body frame velocity and position, allowing computation of estimated interceptor velocity and position , with observed angle of attack and sideslip angle computed from . Estimated body frame position is rotated into the inertial frame using estimated attitude, allowing computation of altitude . Mach is computed from estimated altitude and speed using an interpolated Earth atmospheric model table. Finally, for roll stabilization, the FCS requires an estimate of roll (), which is computed by integrating .
Actuator Model
The channel commanded deflections (pitch, yaw, roll, and speed brake channel commands) from the FCS are converted to physical fin deflections using the fin mix matrix shown in Eq. (12) and the transformations shown in Eqs. (7a) and (14). Note that the speed brake is unused.
is then clipped to remain within the maximum deflection of , and the slew rate is limited to . Each element of is then passed through second order actuator dynamics corresponding to the transfer function shown in Eq. (15), where we use and rad/s (50Hz). The output of the actuator dynamics is then referred to as in the remainder of the document.
Aerodynamic Model
The interceptor geometry is taken from [27] and is shown in Figure 5, with the nominal center of gravity set to -0.54 m. Our nominal aerodynamic model is derived from slender body and Newtonian impact theory [28, 29]. The method in [29] is extended to 6-DOF using a method similar to that in [22], but with a higher fidelity axial force model, rolling moment model, and the addition of a cross channel interference term for control effectiveness. The vehicle diameter is m. The length is m, and the inertia tensor is given in Figure 9. The vehicle mass is 13.5 kg, the reference length is , and the reference area is . The open loop response to flight control commanded deflections are shown in Figure 6 and Figure 7, where are the FCS commanded channel deflections as described in § 2.C.8. We see that the vehicle is slightly open-loop unstable.

Figure 6. Open Loop Airframe Response , Altitude=15 km, speed=2500 m/s

Figure 7. Open Loop Airframe Response , Altitude=25 km, speed=2500 m/s

Figure 9. Environment-Agent Interface
We model aerodynamic uncertainty by randomly modifying the shape of the aerodynamic coefficient curves for each simulated trajectory. To do so, we first bias the coefficients returned from the nominal aerodynamic model as follows. At the start of each episode with probability we uniformly sample an aerodynamic coefficient bias between some bounds , otherwise a coefficient bias is (independently) set to either or with equal probability, noting that in both cases the bias applied to the coefficient groups , , , and are independent. We assume that the bias on and is identical to symmetry (and similarly for and ). Then, for the remainder of the episode, the force and moment coefficients are biased using the sampled value of . We account for CP/CG shifting by modifying the moments as shown in Eqs. (10a) through (10d). Here is uniformly generated at the start of each episode and MRP is the moment reference point, which in this work is set to the CG. Due to symmetry, errors in and will mostly cancel.
Equations of Motion
Let denote the body frame interceptor velocity. The angle of attack and sideslip angle are then computed as shown in Eqs. (11a) through (11b).
The atmospheric density and Mach number are calculated using the 1976 standard Earth atmospheric temperature model. The interceptor control surface deflections , dynamic pressure with , rotational velocity vector , , and are then input to the aerodynamic model described in § 2.D to compute the body frame force and torque .
The rotational velocity is updated by integrating the Euler rotational equations of motion, as shown in Equation (24), where is the interceptor’s inertia tensor, and with the skew symmetric operator defined in Eq. (25).
The interceptor’s attitude is updated by integrating the differential kinematic equations shown in Equation (26), where the interceptor’s attitude is parameterized using the quaternion representation and denotes the component of the rotational velocity vector .
The interceptor body frame velocity is updated by integrating the differential equation given in Eq. (27), where is the interceptor mass, is the DCM mapping from the inertial to body frame given the interceptor attitude , and .
is then rotated into the North-East-Down (NED) inertial frame as shown in Eq. (16a), and is calculated by rotating the vector into the East-North-Up (ENU) reference frame depicted in Figure 2.
The interceptor inertial frame position is then updated by integrating Eq. (30)
The target inertial frame position and velocity unadjusted for drag are then updated by integrating Eqs. (18a) and (32). The target speed is adjusted for drag by integrating Eq. (18c), where at the start of each episode we randomly generate from 0.1 to 0.25 and from 0.05 to 0.10. We use kg.
The target velocity is then computed as shown in Eq. (34)
The equations of motion are updated using fourth order Runge-Kutta integration. For ranges greater than 50 m, a timestep of 5 ms is used, and for the final 50 m of homing, a timestep of 25 us is used in order to more accurately (within 0.125 m) measure miss distance; this technique is borrowed from [30].
3-Loop Autopilot FCS Benchmark
The benchmark FCS (FCS-3L) for this application uses a three-loop autopilot for pitch and yaw control. We assume perfect roll control for the benchmark (and for the augmented benchmark, the FCS-3L-RL), obtained by forcing the -component of rotational velocity to zero, making this a more difficult benchmark to beat. The three-loop autopilot design and gain scheduling are taken from [9], with the modification for an unstable airframe (the linearized airframe is unstable) taken from [31], and modified for our vehicle geometry. This gain scheduling was suboptimal for altitudes above 15 km. Therefore, we performed a series of grid searches on the open-loop crossover frequency , damping , and time constant at three design points: 2500 m/s 15 km, 2500 m/s 20 km, and 2500 m/s 25 km. For a given design point, each design (a tuple of , , ) was simulated for 5000 episodes with up to 10% aerodynamic coefficient bias and 1% center of pressure variation (see § 2.D). We considered a design stable if it resulted in no load constraint violations. We chose this approach as traditional stability metrics such as phase margin with a linearized system can be overly conservative, as a large gain margin can be required to compensate for differences between the linearized and actual system. This approach also gives a better comparison to the FCS-RL, which being inherently nonlinear, cannot be analyzed using linear system metrics. For each FCS parameter tuple at a given design point, we chose the best performing set of parameters that resulted in a stable design. During FCS operation, we interpolate between the parameters at each of the three design points using linear interpolation. This approach is different than what is typically used for interceptor flight control systems, where the gains are optimized at a given design point using step response metrics, but has the advantage that the gains can be optimized considering the dynamics of both the guidance and flight control systems, which are more directly comparable to the approach used for FCS-RL optimization. A block diagram of the pitch and yaw channels for the FCS-3L is illustrated in Figure 8, where , , coming from either the pitch or yaw channel from the guidance system, accelerometer, and rate gyro, respectively, and being either the pitch or yaw commanded channel deflection. The relationship between the block diagram gains and , , can be found in [9]. Step response plots for this benchmark can be found in § 12.A. Figure 13 We did not add adaptive control, as previous work [32] showed that MRCAS was ineffective in modeling this intercept scenario due to the short terminal phase homing time (which is unavoidable with high closing speeds and a realistic seeker range) and the other factors discussed in § 1.

Figure 8. Three-Loop Autopilot Block Diagram

Figure 13. Step Response Comparison: 25 km, 2500 m/s
Meta-RL Optimization Framework
Note that in this section, unless a variable has the "B" (body frame) superscript, it is assumed to be in the LOS stabilized ("S") reference frame.
Overview of Reinforcement Learning
In the reinforcement learning framework, an agent learns through episodic interaction with an environment how to successfully complete a task using a policy that maps observations to actions , i.e., . In actor-critic RL algorithms, policy optimization makes use of a value function that estimates the estimated value of being in state while following policy . Note that the value function can take either ground truth states , observations , or some combination of both, as only the policy is deployed. The environment initializes an episode by generating a ground truth state , mapping this state to an observation, and passing the observation to the agent. The agent uses this observation to generate an action that is sent to the environment; the environment then uses the action and the current ground truth state to generate the next state and a scalar reward signal . The reward and the observation corresponding to the next state are then passed to the agent. The process repeats until the environment terminates the episode, with the termination signaled to the agent via a done signal. In this work, an episode terminates when the closing velocity turns negative, or a path constraint is violated. Trajectories collected over a set of episodes (referred to as rollouts) are collected during interaction between the agent and environment, and used to update the policy and value functions. The interface between agent and environment is depicted in Figure 9, where the environment instantiates the simulator and the system model shown in Figure 1.
Meta-RL differs from generic reinforcement learning in that the agent learns over an ensemble of environments. For example, each environment can contain a unique aerodynamic model, target maneuver model, actuator model, and other peripheral components. Similar to [33], we implement both the policy and value function as a deep recurrent neural network, with each network having a hidden state that is a function of the history of observations . Thus, the policy and value function become and . We optimize the policy using a customized version of the proximal policy optimization algorithm (PPO) [34]. Maximizing the PPO objective function requires learning hidden layer parameters that result in evolving in response to the history of and in a manner that facilitates fast adaptation to environments encountered during optimization, as well as novel environments outside of the training distribution. The optimized policy then adapts real-time to conditions experienced during deployment. Importantly, the network parameters remain fixed during deployment with adaptation occurring through the evolution of .
The network architectures used in this work are shown in Table 2, where is the number of units in layer , obs_dim is the observation dimension, and act_dim is the action dimension. The policy and value functions are periodically updated during optimization after accumulating trajectory rollouts of 30 simulated episodes.
| Policy Network | Value Network | ||||
| Layer | # units | activation | # units | activation | |
| hidden 1 | tanh | tanh | |||
| hidden 2 | tanh | tanh | |||
| hidden 3 | tanh | 5 | tanh | ||
| output | linear | 1 | linear |
Table 2. Policy and Value Function network architecture
Meta-RL Problem Formulation for Gain Adaptation (FCS-3L-RL)
The agent observation is shown in Eqs. (35), where , , , , , and are generated as shown Sections II.C.3, II.C.4, and II.C.6. are the commanded channel deflections from Section II.C.8 and derived from policy actions as described in the following. are the adaptive gain biases that modify the scheduled gains for the three-loop FCS. These biases are integrated from the policy actions, and must be included in the observation space. The observations are scaled to have zero mean and unit variance.
The output of the policy is scaled through multiplication by and input to an integrator that saturates at . The integrator output is then split into two vector components, each in , with each set biasing the pitch and yaw channel three loop autopilot gains. The integration improves performance by smoothing the action. We use and
The reward function is shown below in Equations (21a) through (40). is a shaping reward given at each step in an episode. These shaping rewards take the form of a Gaussian-like function of the norm of the scaled acceleration tracking error as shown in Eq. (21a). is a control effort penalty, again given at each step in an episode, and is a bonus given at the end of an episode if the miss distance is below the threshold . Importantly, the current episode is terminated if a path constraint is violated, in which case the stream of positive shaping rewards is terminated, and the agent receives a negative reward. We use , , , , , m, and . We use a discount rate of 0.90 for shaping rewards and 0.995 for the terminal reward.
The gain definitions are from [9], and defined in Table 3. Figure 10 illustrates the gain adaptation during an engagement with a jinking target. Note that the occasional bias saturation indicates that further performance improvement may have been possible with further hyperparameter tuning.
| Gain | Definition |
| Steady State | |
| Acceleration feedback | |
| Forward path integrator | |
| Rate feedback | |
| Aerodynamic acceleration | |
| Aerodynamic body rate |
Table 3. Gain Definitions

Figure 10. Adaptive Gain Biases over Engagement
For optimization, aerodynamic uncertainty is modeled using a maximum coefficient bias of 10% and a maximum center of pressure shift equal to 1% (see § 2.D). A learning curve plotting the mean reward over rollouts versus episode is shown in Figure 11. Here each optimization rollout consists of 30 simulated episodes. The reward history is slightly noisy due to the fact that the exploration variance does not fall to zero during the optimization (it would be destabilizing to the PPO algorithm if it did fall to zero). Note that for testing, the exploration variance is disabled, and we return the mean value of the policy distribution. The maneuver distribution used for both optimization and testing is 33% step, 33% weave, and 34% jinking.

Figure 11. Optimization Reward History for FCS-3L-RL
Meta-RL Problem Formulation, Full FCS (FCS-RL)
The agent observation is shown in Eqs. (41), where , , , , , and are generated as shown Sections II.C.3, II.C.4, and II.C.6. is obtained through integration of the component of the measured rotational velocity () and are the commanded channel deflections from Section II.C.8 and derived from policy actions as described in the following. The observations are scaled to have zero mean and unit variance.
The output of the policy is split into three scalar components, a pitch channel deflection rate , a yaw channel deflection rate , and a roll channel deflection rate . These are scaled by multiplication with the maximum allowed deflection rate and then clipped to fall within the allowed commanded deflection rates. The maximum deflection rates are 500°/s. These commanded deflection rates are then integrated, with the integrator saturating at the maximum allowed deflection, giving the commanded deflections , , and
The reward function is shown below in Equations (23a) through (23f). is a shaping reward given at each step in an episode. These shaping rewards take the form of a Gaussian-like function of the norm of the scaled acceleration tracking error as shown in Eq. (36). encourages the agent to minimize deviations of body roll from zero, with differential fin deflections only used to counteract the rolling moment induced by angle of attack and sideslip (See § 2.D). is a control effort penalty, again given at each step in an episode, and is a bonus given at the end of an episode if the miss distance is below the threshold . Importantly, the current episode is terminated if a path constraint is violated, in which case the stream of positive shaping rewards is terminated, and the agent receives a negative reward. We use , , , , , m, , and . We use a discount rate of 0.90 for shaping rewards and 0.995 for the terminal reward.
We use curriculum learning [35] to optimize a policy that adapts to high levels of aerodynamic uncertainty. Specifically, we first optimize the policy using coefficient bias bounds (CB) of 10% and center of pressure location bounds (CP) of 1% (see § 2.D). The resulting policy is then re-optimized using CB=20% and CP=2%, but with the policy initial exploration standard deviation set to 0.30 (as opposed to 1.00 for the initial optimization), allowing the second optimization run to retain learning from the first optimization, but giving the policy enough exploration ability to improve performance under the higher aerodynamic uncertainty. We then repeat this incremental optimization a third time, using CB=40% and CP=2%. This results in improved adaptability to aerodynamic uncertainty compared to trying a single full optimization using CB=40% and CP=2%. The reward learning curve for the initial optimization is shown in Figure 12, where "Mean" is the mean reward over a set of rollouts. Before episode 10000, the agent is primarily learning to avoid constraint violations, which result in early termination of an episode. The maneuver distribution used for both optimization and testing is 33% step, 33% weave, and 34% jinking.

Figure 12. Optimization Reward History
Experiments
FCS Relative Performance
Relative performance is measured by running 5000 simulations for each case in the following tables. In the following tables, "AREF" (in g) is the reference acceleration (see § 2.B) for the target, "CB" is the aerodynamic coefficient bias, "CP" is the aerodynamic center of pressure shift (see § 2.D), and "R" is the maximum radome refraction in radians (see § 2.C.1). The FCS-RL, FCS-3L, and FCS-3L-RL use the labels ’RL’, ’3L’, and ’3L-A’, respectively.
Relative performance is quantified using several metrics. Accuracy is quantified by "Miss < 25 cm", which indicates the percentage of episodes that resulted in the interceptor achieving a miss distance less than or equal to 25 cm. "IR" is the interceptor ratio, which denotes the number of interceptors using the FCS-3L and FCS-3L-RL that are required to match the accuracy of an interceptor using the FCS-RL, according to the equation
where and are the probability of a miss distance greater than 25 cm for the FCS-RL and either FCS-3L or FCS-3L-RL, respectively. Control activity is measured using the root-mean-square of the magnitude of the interceptor acceleration "RMS ". Finally, "Vio (%)" gives the percentage of episodes that resulted in a load constraint violation. Table 4 shows that under conditions of no radome refraction, no aerodynamic uncertainty, and no target evasive acceleration, there is essentially no performance difference between the three flight control systems. To demonstrate the need for a recurrent policy in this application, we optimized a version of FCS-RL with the recurrent layer replaced with a standard feed forward layer (RL-MLP). The results clearly demonstrate the need for a recurrent policy (which allows realization of an integrator) when applying RL to flight control over a large flight envelope.
| Case | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| AREF=0 CB=0.0-CP=0.0 R=0 | 3L | 100 | 1.0 | 41 | 0.0 |
| AREF=0 CB=0.0-CP=0.0 R=0 | 3L-RL | 100 | 1.0 | 41 | 0.0 |
| AREF=0 CB=0.0-CP=0.0 R=0 | RL | 100 | - | 40 | 0.0 |
| AREF=0 CB=0.0-CP=0.0 R=0 | RL-MLP | 6 | - | 144 | 34.1 |
Table 4. Ideal Conditions
Table 5 tabulates performance when we consider radome refraction (§ 2.C.1). Radome refraction results in parasitic interaction between the guidance and flight control systems that reduces performance. Except for the special case of a hemispherical radome, the actual LOS refraction depends on where the LOS passes through the radome. Furthermore, in order for the FCS to track a commanded acceleration, an interceptor using control surface actuation must rotate. This rotation results in the LOS to the target passing through a different part of the radome, causing the refraction angle to change, resulting in a false sense of target motion. The guidance system then reacts to this by giving a commanded acceleration that takes into account the false target motion, completing the parasitic feedback loop. We see that even with small radome refraction a performance differential appears between the flight control systems, with the differential growing as the radome refraction is increased.
| Case | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| AREF=4 CB=0.10-CP=0.01 R=0.0005 | 3L | 70 | 2.5 | 61 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.0005 | 3L-A | 80 | 1.9 | 59 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.0005 | RL | 95 | - | 56 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.001 | 3L | 60 | 2.5 | 60 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.001 | 3L-A | 70 | 1.9 | 59 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.001 | RL | 90 | - | 56 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.002 | 3L | 35 | 2.7 | 63 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.002 | 3L-A | 45 | 2.0 | 62 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.002 | RL | 69 | - | 57 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.005 | 3L | 11 | 2.7 | 66 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.005 | 3L-A | 15 | 1.9 | 65 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 R=0.005 | RL | 27 | - | 63 | 0.0 |
Table 5. Radome Sensitivity
For the simulations used to create Table 6 we increase AREF from 4 to 8, doubling the target’s acceleration at a given dynamic pressure. Again, there is a clear difference between the performance of the three flight control systems. Next, Table 7 explores which target maneuvers are more difficult for the flight control systems. Whereas the performance for all flight control systems is nearly identical for the step maneuver, we see that a large performance differential exists for weaving and jinking maneuvers. Importantly, these types of maneuvers are likely to be employed by a target that is actively maneuvering to avoid interception, as it keeps the threat from deviating too far from the trajectory required to hit its target. Moreover, it is possible for certain types of reentry vehicles to spiral even without the capability of actively evading interception [36, 37].
| Case | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| AREF=8 CB=0.10-CP=0.01 | 3L | 52 | 2.4 | 95 | 0.0 |
| AREF=8 CB=0.10-CP=0.01 | 3L-A | 53 | 2.3 | 96 | 0.0 |
| AREF=8 CB=0.10-CP=0.01 | RL | 83 | - | 91 | 0.0 |
Table 6. Increased Target Evasive Acceleration (R=0.0005)
| Case | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| Step | 3L | 98 | 1.2 | 65 | 0.0 |
| Step | 3L-A | 97 | 1.3 | 65 | 0.0 |
| Step | RL | 99 | - | 46 | 0.0 |
| Weave | 3L | 47 | 4.7 | 54 | 0.0 |
| Weave | 3L-A | 65 | 2.9 | 52 | 0.0 |
| Weave | RL | 95 | - | 47 | 0.0 |
| Jink | 3L | 66 | 2.7 | 63 | 0.0 |
| Jink | 3L-A | 75 | 1.7 | 61 | 0.0 |
| Jink | RL | 90 | - | 57 | 0.0 |
Table 7. Target Maneuver Sensitivity (AREF=4, CB=0.10, CP=0.01, R=0.0005)
Table 8 reports performance with various target weave periods, ranging from 0.1 to 1.0 seconds. We see that the weave period that maximizes miss distance is different between the three flight control systems. To see why, consider that in [38] an expression is derived showing peak miss distance as a function of normalized target weave frequency, specifically , where and are the target weave frequency and guidance system time constant, respectively. When taking into account FCS dynamics, the peak miss distance becomes a function of the effective time constant of the combined guidance and control system. Indeed, for a given guidance system, a decrease in flight control response time can improve effectiveness against weaving targets [38]. Importantly, although all three flight control system’s performance are impacted by weaving targets, there is again a clear performance gap between the three systems. The performance difference is at least partly due to the improved flight control response time of the FCS-RL (See Figure 13 in the following). It is also possible that the FCS-RL is dynamically adapting its effective bandwidth to increase performance.
| Weave Period (s) | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| 0.1 | 3L | 100 | 1.0 | 42 | 0.0 |
| 0.1 | 3L-A | 100 | 1.0 | 41 | 0.0 |
| 0.1 | RL | 100 | - | 40 | 0.0 |
| 0.50 | 3L | 50 | 1.4 | 45 | 0.0 |
| 0.50 | 3L-A | 45 | 1.7 | 48 | 0.0 |
| 0.50 | RL | 63 | - | 45 | 0.0 |
| 1.00 | 3L | 24 | 7.7 | 53 | 0.0 |
| 1.00 | 3L-A | 30 | 5.9 | 53 | 0.0 |
| 1.00 | RL | 88 | - | 45 | 0.0 |
Table 8. Target Weave Period (AREF=4, CB=0.10, CP=0.01, R=0.0005)
Table 9 illustrates the impact of increasing the level of aerodynamic uncertainty. We see that although all three FCS are robust to aerodynamic uncertainty, the performance advantage of both the FCS-RL and FCS-3L-RL are maintained as compared to FCS-3L, with a higher hit rate, lower control effort, and fewer constraint violations Finally, Table 10 gives interceptor and target statistics computed over 5000 trajectories, where we see that the acceleration of the interceptor and target are quite close at the end of the engagement, making this a highly agile target. Since the target starts at a much higher altitude than the interceptor, the lower dynamic pressure gives the target less maneuverability earlier in the engagement.
| Cumulative RMS (m/s) | 40 |
| Cumulative RMS (m/s) | 12 |
| Terminal RMS (m/s) | 58 |
| Terminal RMS (m/s) | 51 |
| Maximum Terminal Altitude (m) | 31516 |
| Minimum Terminal Altitude (m) | 12838 |
Table 10. Typical High Altitude Engagement Statistics
| Case | FCS | Miss < 25 cm (%) | IR | RMS (m/s) | Vio (%) |
| AREF=4 CB=0.10-CP=0.01 | 3L | 70 | 2.5 | 61 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 | 3L-A | 80 | 1.9 | 59 | 0.0 |
| AREF=4 CB=0.10-CP=0.01 | RL | 95 | - | 56 | 0.0 |
| AREF=4 CB=0.20-CP=0.02 | 3L | 66 | 2.5 | 62 | 0.6 |
| AREF=4 CB=0.20-CP=0.02 | 3L-A | 77 | 1.8 | 60 | 0.4 |
| AREF=4 CB=0.20-CP=0.02 | RL | 93 | - | 57 | 0.0 |
| AREF=4 CB=0.40-CP=0.03 | 3L | 60 | 1.9 | 65 | 3.4 |
| AREF=4 CB=0.40-CP=0.03 | 3L-A | 70 | 1.5 | 63 | 3.2 |
| AREF=4 CB=0.40-CP=0.03 | RL | 84 | - | 60 | 0.8 |
Table 9. Aerodynamic Uncertainty (R=0.0005)
Step response plots at 25 km altitude and 2500 m/s comparing the FCS-RL and FCS-3L under nominal, "slow", and "fast" aerodynamic models are shown in Figure 13. The slow model has a reduction in aerodynamic force and moment coefficients and a shift in CP giving increased static margin, whereas the fast model has an increase in aerodynamic coefficients and reduced static margin. Although this is modeled as a shift in CP, it would be equivalent to a shift in CG in the opposite direction. We see that the FCS-RL exhibits a significant reduction in flight control response time as compared to the benchmark. A full trajectory for the FCS-RL starting at 25 km altitude with inclination to target is shown in Figure 14.

Figure 14. FCS-RL sample trajectory
Policy Generalization Testing
We demonstrate the generalization capability of the three flight control systems by modifying the engagement initial conditions, as shown in Table 11. Note that in any real application, the homing phase is effective over some "basket" of initial conditions consistent with the end of the mid-course guidance phase. This basket is limited by a minimum range-to-target that gives the interceptor enough time to complete a successful terminal phase intercept, with the minimum time being a function of target maneuver, closing speed, and initial heading error. Using these modified initial conditions, we test the same target maneuver distribution as in § 12.A, and also test against two target maneuver types not seen during optimization, a vertical S maneuver [39] (vert-S) and frequency modulated weave (FM-weave). The vertical S maneuver is identical to the weave maneuver except that the target acceleration instantaneously transitions between minimum and maximum acceleration at the weave maneuver zero acceleration crossings. In our simulator the vertical S differs from the jinking maneuver in that the jinking maneuver does not switch sign with a fixed period, and the jinking is independent for each axis. The frequency modulated weave has a random variation in the weave frequency over each engagement. This would be a relatively simple maneuver to execute for a maneuverable re-entry vehicle that is already spiraling with the fin deflections used to modulate the spiraling frequency.
| Elevation Angle to Target (degrees) | 30 | 60 |
| Azimuth Angle to Target (degrees) | -180 | 180 |
| Altitude (m) | 15000 | 22000 |
| Missile Velocity Magnitude (m/s) | 2200 | 2800 |
| Initial Range to Targeted Asset (m) | 15000 | 30000 |
| Target Velocity Magnitude (m/s) | 2500 | 4500 |
| Target Velocity Cone Half Apex Angle (degrees) | 0 | 20 |
Table 11. Simulation Modified Initial Conditions)
Example target acceleration trajectories for the FM-weave maneuver are illustrated in Figure 15.

Figure 15. Frequency Modulated Weave Sample Trajectories
From Table 12 we see that all three flight control systems are robust to an increase in both the range of initial conditions and uncertainty in the initial state estimate. The vertical-S maneuver is challenging for all three flight control systems, but more so for the FCS-3L and FCS-3L-RL. The key takeaway is that the RL FCS policy generalizes well to both initial conditions and target maneuvers outside of the optimization distribution. The FCS-RL performance is not affected by the FM-weave maneuver, whereas the FCS-3L and FCS-3L-RL are significantly reduced.
| Case | UKF | Miss < 25cm (%) | RMS (m/s) | IR | Vio (%) |
| Mod-IC | 3L | 73 | 49 | 1.9 | 0.0 |
| Mod-IC | 3L-A | 83 | 49 | 1.4 | 0.0 |
| Mod-IC | RL | 92 | 46 | - | 0.0 |
| Mod-IC vert-S | 3L | 42 | 49 | 2.2 | 0.0 |
| Mod-IC vert-S | 3L-A | 47 | 58 | 1.9 | 0.0 |
| Mod-IC vert-S | RL | 70 | 45 | - | 0.0 |
| Mod-IC FM-weave | 3L | 58 | 47 | 2.9 | 0.0 |
| Mod-IC FM-weave | 3L-A | 54 | 52 | 3.3 | 0.0 |
| Mod-IC FM-weave | RL | 92 | 43 | - | 0.0 |
Table 12. Generalization Cases
Flight Certification
Linear system theory can provide theoretical stability guarantees for the FCS-3L in the simplified case of a linearized system. However, optimal gain scheduling requires evaluating performance in a more realistic non-linear simulator [40]. Importantly, it is possible that for the case of a non-linear system, the system can be unstable at some flight condition while at the same time passing stability checks using the linearized system. This becomes more likely as the aerodynamics become more non-linear (which can be the case in the hypersonic flight regime). In addition, the actual deployment aerodynamics can be quite different from the simulated aerodynamics, weakening stability arguments based on sufficient gain or phase margin. Consequently, the FCS is not actually provably stable using linear systems theory when coupled with non-linear aerodynamics and significant aerodynamic uncertainty. Although a sufficiently large gain margin could compensate for differences between the actual and linearized systems, this comes with a performance cost. Consequently, extensive Monte Carlo simulation is required before moving to a flight test, which is also possible using the FCS-RL optimization process. Importantly, there has been recent work on the certification of stability for a meta-RL policy [41]. Finally, although the FCS-3L-RL underperformed the FCS-RL, it has the advantage of allowing flight certification using linear stability metrics, which could easily be implemented by limiting the gain bias to ensure that the resulting gains result in adequate margin for the linear system. This would allow a similar flight certification procedure as current systems.
Conclusion
We applied reinforcement meta-learning (meta-RL) to two approaches to flight control system (FCS) optimization. In the first approach (FCS-RL) we optimize an adaptive policy directly mapping commanded acceleration tracking error and sensor outputs to commanded fin deflection rates in the pitch, yaw, and roll channels. In the second approach (FCS-3L-RL), the scheduled gains for a standard FCS (FCS-3L) using the three-loop autopilot for pitch and yaw control are adaptively adjusted during a trajectory using a meta-RL gain adaptation policy. The three flight control systems were then extensively tested in a 6-DOF simulator that modeled most factors relevant to flight control. Under ideal conditions, the performance of the three flight control systems was nearly identical. However, under conditions of radome refraction, target evasive maneuvers, and aerodynamic uncertainty, we found a significant performance differential appear in favor of the FCS-RL, with higher interception rates and lower control effort. We believe that there are two primary factors behind the FCS-RL performance. The first is the higher expressive power of the recurrent policy compared to a fixed three-loop topology, which allows for faster flight control response time, as measured by step response rise time. The second is the adaptability of the FCS-RL achieved by optimizing over an ensemble of aerodynamic models, and improved through curriculum learning. It is likely that the integrated pitch, yaw, and roll channel control has a smaller effect, but would have a much greater impact in bank-to-turn applications where aerodynamic coupling is higher. To put the performance differential in perspective, facing an actively evading target under small levels of radome refraction and aerodynamic uncertainty, the interception rate for the FCS-RL and FCS-3L are 95% and 70% respectively. This implies that 2.5 interceptors using the benchmark system would be required to reach the FCS-RL hit rate for a single interceptor. In other more challenging scenarios, the FCS-3L required more than 5 interceptors to match the FCS-RL performance. In addition to the obvious cost advantage, this increased accuracy can be critical in scenarios with a limited number of interceptors and/or launchers. Although FCS-3L-RL also demonstrates improved performance compared to FCS-3L-RL, the performance differential is smaller than FCS-RL. Finally, we demonstrate that both the FCS-3L-RL and the FCS-RL generalize well to initial conditions and target maneuvers that were not experienced during optimization.
References
- [1]Alviani, R., Poggie, J., and Blaisdell, G., “Unsteady Aspects of Shock-Wave/Boundary-Layer Interaction Resulting from Control Surface Deflection,” AIAA Journal, Vol. 60, No. 8, 2022, pp. 4649–4659. https://doi.org/10.2514/1.J061337
- [2]Edwards, J. R., “Numerical Simulations of Shock/Boundary Layer Interactions Using Time-Dependent Modeling Techniques: A Survey of Recent Results,” Progress in Aerospace Sciences, Vol. 44, No. 6, 2008, pp. 447–465. https://doi.org/10.1016/j.paerosci.2008.06.004
- [3]Witeof, Z., and Pasilaio, C. L., “Fluid-Thermal-Structural Interaction Effects in Preliminary Design of High Speed Vehicles,” 56th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 2015. https://doi.org/10.2514/6.2015-1631
- [4]Falkiewicz, N. J., Cesnik, C. E. S., Bolender, M. A., and Doman, D. B., “Thermoelastic Formulation of a Hypersonic Vehicle Control Surface for Control-Oriented Simulation,” AIAA Guidance, Navigation, and Control Conference, 2009. https://doi.org/10.2514/6.2009-6284
- [5]Culler, A. J., and McNamara, J. J., “Impact of Fluid-Thermal-Structural Coupling on Response Prediction of Hypersonic Skin Panels,” AIAA Journal, Vol. 49, No. 11, 2011, pp. 2393–2406. https://doi.org/10.2514/1.J050617
- [6]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 126–132. https://doi.org/10.2514/4.868948
- [7]Siouris, G. M., “Missile guidance and control systems,” Springer Science & Business Media, 2004, pp. 142–143. https://doi.org/10.1007/b97614
- [8]Shneydor, N. A., “Missile guidance and pursuit: kinematics, dynamics and control,” Elsevier, 1998, pp. 151–155. https://doi.org/10.1533/9781782420590.
- [9]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 529–564. https://doi.org/10.2514/4.868948
- [10]Stallard, D. V., “An Approach to Autopilot Design for Homing Interceptor Missiles,” AIAA Guidance, Navigation, and Control Conference, AIAA, 1991, pp. 99–113.DOI
- [11]Garnell, P., Guided Weapon Control Systems, 2nd ed., Pergamon Press, Oxford, 1980.DOI
- [12]Lee, H., Snyder, S., and Hovakimyan, N., “L₁ Adaptive Output Feedback Augmentation for Missile Systems,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 54, No. 2, 2018, pp. 680–692. https://doi.org/10.1109/TAES.2017.2764218
- [13]Rusnak, I., Weiss, H., and Barkana, I., “Improving the Performance of Existing Missile Autopilot Using Simple Adaptive Control,” International Journal of Adaptive Control and Signal Processing, Vol. 28, No. 7-8, 2014, pp. 732–749. https://doi.org/10.1002/acs.2457
- [14]Joshi, S. M., Tao, G., and Khong, T., “A Direct Adaptive Control Approach in the Presence of Model Mismatch,” AIAA Guidance, Navigation, and Control Conference, AIAA, Chicago, Illinois, 2009. https://doi.org/10.2514/6.2009-5620
- [15]Kannan, S. K., and Johnson, E. N., “Model Reference Adaptive Control with a Constrained Linear Reference Model,” 49th IEEE Conference on Decision and Control, IEEE, 2010, pp. 48–53. https://doi.org/10.1109/CDC.2010.5717275
- [16]Rohrs, C. E., Valavani, L., Athans, M., and Stein, G., “Robustness of Continuous-Time Adaptive Control Algorithms in the Presence of Unmodeled Dynamics,” IEEE Transactions on Automatic Control, Vol. 30, No. 9, 1985, pp. 881–889. https://doi.org/10.1109/TAC.1985.1104070
- [17]Cao, C., and Hovakimyan, N., “Design and Analysis of a Novel L₁ Adaptive Control Architecture with Guaranteed Transient Performance,” IEEE Transactions on Automatic Control, Vol. 53, No. 2, 2008, pp. 586–591. https://doi.org/10.1109/TAC.2007.914282
- [18]Shin, H.-S., He, S., and Tsourdос, A., “A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design,”, 2019. URL https://arxiv.org/abs/1908.06884
- [19]Taligatos, R., Nguyen, D. H., and Lowenberg, M. H., “Investigating Missile Feedback Control: Reinforcement Learning, PID, or Both?” AIAA SCITECH 2026 Forum, AIAA, 2026. https://doi.org/10.2514/6.2026-1162
- [20]Candeli, A., De Tommasi, G., Lui, D. G., Mele, A., Santini, S., and Tartaglione, G., “A Deep Deterministic Policy Gradient Learning Approach to Missile Autopilot Design,” IEEE Access, Vol. 10, 2022, pp. 19685–19696. https://doi.org/10.1109/ACCESS.2022.3150926
- [21]Wang, W., Wu, M., Chen, Z., and Liu, X., “Integrated Guidance-and-Control Design for Three-Dimensional Interception Based on Deep-Reinforcement Learning,” Aerospace, Vol. 10, No. 2, 2023, p. 167. https://doi.org/10.3390/aerospace10020167
- [22]Gaudet, B., and Furfarо, R., “A Comparison of Partially and Fully Integrated Guidance and Flight Control Optimized with Reinforcement Meta-Learning,” AIAA SCITECH 2023 Forum, 2023, p. 1628.DOI
- [23]Gaudet, B., Charcos, I., and Furfaro, R., “Integrated and Adaptive Guidance and Control for Endoatmospheric Missiles via Reinforcement Learning,” arXiv preprint arXiv:2109.03880, 2021.DOI
- [24]Gaudet, B., Durant, D., and Furfaro, R., “Reinforcement Learning for Target Jerk Estimation in Interceptor Homing Phase,”, 2026. URL https://arxiv.org/abs/2611.15923
- [25]Gaudet, B., Furfaro, R., Linares, R., and Scorsoglio, A., “Reinforcement Metalearning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop,” Journal of Spacecraft and Rockets, Vol. 58, No. 2, 2021, pp. 386–399.DOI
- [26]Shneydor, N. A., “Missile guidance and pursuit: kinematics, dynamics and control,” Elsevier, 1998, p. 118. https://doi.org/10.1533/9781782420590
- [27]Vasile, J. D., Bryson, J. T., Sahu, J., Paul, J. L., and Gruenwald, B. C., “Aerodynamic Dataset Generation of a Long-Range Projectile,” Tech. rep., ARMY RESEARCH LAB ABERDEEN PROVING GROUND MD ABERDEEN PROVING GROUND United . . ., 2020.
- [28]Jorgensen, L. H., “A method for estimating static aerodynamic characteristics for slender bodies of circular and noncircular cross section alone and with lifting surfaces at angles of attack from 0 deg to 90 deg,” 1973.
- [29]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 473–482. https://doi.org/10.2514/4.868948
- [30]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 18–21. https://doi.org/10.2514/4.868948
- [31]Zarchan, P., “Advanced Tactical and Strategic Missile Guidance, Volume 2,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 1–29. https://doi.org/10.2514/4.105388
- [32]Furfaro, R., and Gaudet, B., “Deep Learning Framework for Rapid development of High Performance Hypersonic Weapons, 2021-002 T2, GN&C Working Group Final Report,” University Consortium for Applied Hypersonics, Contract: HQ0034229CA10, 2026. Final report is CUI, contact UCAH for access. Report will appear in DTIC. PI: Furfaro, Working group lead: Gaudet.
- [33]Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M., “Learning to reinforcement learn,” arXiv preprint arXiv:1611.05763, 2016.arxiv.org/abs/1611.05763
- [34]Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
- [35]Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., and Stone, P., “Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey,” Journal of Machine Learning Research, Vol. 21, No. 181, 2020, pp. 1–50.arxiv.org/abs/2003.04960
- [36]Chadwick, W. R., and Zarchan, P., “Interception of Spiraling Ballistic Missiles,” Proceedings of the American Control Conference, IEEE, Seattle, Washington, USA, 1995, pp. —. Discusses guidance and interception performance against spiraling ballistic reentry vehicles.
- [37]Platus, D. H., “Ballistic Re-entry Vehicle Flight Dynamics,” Journal of Guidance and Control, Vol. 5, No. 1, 1982, pp. 4–16. https://doi.org/10.2514/3.56131, analysis of ballistic reentry vehicle aerodynamic and rotational dynamics during atmospheric reentry.
- [38]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 439–471. https://doi.org/10.2514/4.868948
- [39]Zarchan, P., “Tactical and strategic missile guidance,” American Institute of Aeronautics and Astronautics, Inc., 2012, pp. 120–121. https://doi.org/10.2514/4.868948
- [40]Jackson, P. B., “Overview of Missile Flight Control Systems,” Johns Hopkins APL Technical Digest, Vol. 29, No. 1, 2010, pp. 9–24. URL https://www.jhuapl.edu/sites/default/files/2024-09/29-01-Jackson.pdf
- [41]Scorsoglio, A., D’Ambrosio, A., and Furfaro, R., “[Article Title],” Journal of Guidance, Control, and Dynamics, 2026, pp. 1–17. https://doi.org/10.2514/1.G009769 URL https://doi.org/10.2514/1.G009769 articles in Advance.