WorksComputer Science
AGAMOTTO: An AI-Enabled Framework for Time-Critical Tactical Course-of-Action Evaluation and Generation
Abstract
Current command and control (C2) platforms lack integrated simulation capabilities that would allow commanders to dynamically explore courses of action (COAs) during live operations. This paper presents ongoing research and contributions to the development of AGAMOTTO (Advanced Guidance and Analytics for Military Operations, Tactical Threats, and Outcomes) as part of the overarching NGC2 program, which advances the Army's efforts to modernize C2 systems, such as the Tactical Assault Kit (TAK). The paper outlines the current design, preliminary findings, and the remaining research tasks, with an emphasis on how the completed work will support tactical decision-making in future time-constrained environments. While the full system is still under development, initial development and results demonstrate the feasibility of AGAMOTTO. Early experiments highlight both the potential benefits and practical constraints of deploying AI-enabled tools in operational settings, and introduce lessons learned and a viable training strategy to make AI-generated COAs feasible in tactical scenarios.
Introduction
Modern military operations require rapid, data-driven decision-making under conditions of uncertainty. These challenges are likely to become more acute as both the enemy and friendly decision processes become driven by artificial intelligence tools[1], and decision timelines compress. At the tactical level, with limited resources and manpower, the ability to adapt to shortened decision timelines is hindered without access to AI-enabled decision support systems (DSSs). This creates an environment in which tactical decision-makers are at a disadvantage and require AI-integrated tools to overcome this disadvantage.
One potential avenue to integrate AI-enabled DSS into tactical operations is during planning. Tactical leaders must receive operational orders and synthesize them into a coherent, actionable plan. In the U.S. Army, at the company level and below, this process is called the Troop Leading Procedures (TLP) and requires a tactical leader to produce an order to their subordinate troops, including a plan to accomplish their mission and objectives. Their mission includes tactical tasks, specific requirements that outline the exact outcome desired with respect to friendly troops, the enemy, and the given terrain. This process can be labor-intensive, with little assistance to the tactical leader outside their own training and that of their subordinates.
Integrating AI into this planning process can provide tactical leaders with advantages by reducing planning time and enabling task-focused optimization of outcomes. A directly accessible simulation environment could enable tactical leaders to digitally execute their plan without risking lives or other resources and to gain feedback before the start of the mission. Leveraging AI methods such as reinforcement learning (RL), this simulation environment could also serve as a platform for AI agents to explore potential solutions in search of an optimal outcome and recommend the corresponding path to the leader, thereby reducing cognitive and time requirements. However, no tools exist that can dynamically model the requirements of the tactical mission, including the specific terrain and enemy, focusing on the assigned tactical task. Such a tool would also need to be accessible to tactical leaders (i.e., integrated into their existing tactical systems) and generate recommended solutions within the unit’s planning timelines, sometimes measured only in a few hours.
Partly to address gaps such as this, the then Army Futures Command (AFC) commissioned the Next Generation C2 platform (NGC2) [5]. Of the nine key characteristics outlined in the document, NGC2 “must also include an organic simulation tool . . .” This paper outlines ongoing research and contributions to a tool to address these limitations as part of the overarching NGC2 program, which supports the Army’s efforts to modernize C2 systems. This tool, named the Advanced Guidance and Analytics for Military Operations, Tactical Threats, and Outcomes (AGAMOTTO) system, is designed to create a real-time feedback loop between an arbitrary C2 system (i.e., the NGC2 platform) and existing Army-funded simulation environments (i.e., One Semi-Automated Forces (OneSAF)) for: 1) simulation of planned Army operations to aid in rehearsals and planning, and 2) predictive analysis of current tactical situations by simulating and visualizing potential enemy actions based on known or templated enemy situation.
In this paper, we outline the AGAMOTTO system, and discuss the limits of existing RL techniques applied to similar problem sets. Additionally, we outline other state-of-the-art RL methods that may be more generally applicable to the mission and time requirements in tactical operations. Specifically, this paper introduces:
A novel methodology for integrating simulation and AI into currently fielded tactical C2 systems to improve tactical outcomes and planning.
Initial evaluation results for a task-focused RL reward system, training AI agents to optimize tactical outcomes according to specific mission requirements.
A discussion of the current technical difficulties of such systems, and initial work to overcome them.
An outline for a way forward to fully develop and integrate such a system into tactical C2 systems, such as NGC2
Related Works and Key Insights
AI-Integrated Decision Support Systems
Due in no small part to White House guidance, the Department of War (DoW) has accelerated programs to integrate AI in military DSS [19, 20]. Projects such as Thunderforge support strategic, theater-level planning through large language models and battle management systems, such as Anduril’s Lattice [6]. Other projects have been linked to strategic and theater targeting, as well as battlespace management [4, 15]. These systems are designed to assist in large-scale planning efforts led by senior commanders and large staffs. The U.S. Army War College has experimented with AI for wargaming, a method for evaluating the efficacy of plans before execution, but this is generally not done at the tactical level [25]. There has been very little research on integrating simulation and optimization capabilities at the tactical level, creating a gap in AI integration efforts across the operational force.
Reinforcement Learning for Tactical Optimization
RL has been identified as a potential avenue to address the gap in AI-integrated tactical DSS [1]. RL has achieved breakthroughs in complex decision-making in games, achieving superhuman outcomes [16, 23, 14, 11].
RL is an AI learning framework in which an agent learns to select actions in an environment in order to maximize cumulative reward. An RL problem is typically formalized as a Markov Decision Process (MDP) , where:
is the set of states,
is the set of actions,
is the transition probability,
is the expected immediate reward,
is the discount factor.
At each time step , the agent observes a state , selects an action according to a policy , receives a reward , and transitions to a new state according to .
The objective of RL is to find an optimal policy that maximizes the expected discounted return
In our use case, a state could be a set of friendly and enemy positions. Actions could include movement (i.e., a single “step” of a fraction of a meter in a given direction), firing a weapon, or other tactical actions. An optimal policy would allow an agent to traverse a digital environment in such a way to maximize their rewards, defined arbitrarily but likely tied to control of terrain, destruction of enemies, and loss of friendly troops.
Recent work has sought to leverage this technology for military applications, using high-fidelity military simulations as training environments [7, 2]. Goecks et al. have even attempted to integrate mission tasks into the reward system for RL training, demonstrating the feasibility and advantages of this approach [8]. However, most work, including work by Goecks et al., uses pixel data from the simulation itself for training. This is infeasible in a live tactical environment, such as our use case, because it can only train on what is shown on the simulation screen rather than on a higher-fidelity method, such as a combat simulator. This distance between the real world and the simulation highlights the so-called “sim-to-real” gap faced in modern military simulations [12].
Key Insights
Military simulators, such as the Rapid Integration and Development Environment (RIDE), can provide a simulation and training platform that uses realistic combat physics and terrain [10]. This can overcome the lack of high-fidelity training environments for RL agents and address the gap in AI-enabled tactical DSS. When combined with RL packages, these simulation environments can replicate the tactical leader’s real-world conditions and enable RL to explore realistic terrain, enemy engagements, and friendly actions. When combined with information from existing tactical DSS, such as the Tactical Assault Kit (TAK), which is widely fielded today, this training backbone can optimize outcomes against the enemy envisioned in the tactical leader’s order, on the actual terrain, and informed by the leader’s tactical objectives and tasks.
Using these insights, we now seek to outline a system for AI-integrated tactical DSS, which we call AGAMOTTO.
Technical Requirements
AGAMOTTO is envisioned as a prototype, proof-of-concept system that provides tactical leaders with access to a simulation and COA generation tool as part of C2 system functionality, as envisioned in NGC2, to aid in the planning, evaluation, and execution of tactical operations. As NGC2 is still under active development, AGAMOTTO must be modular, providing a platform-agnostic method to encapsulate and transfer data between arbitrary C2 systems and simulation environments. Figure 1 gives a high-level overview of this design. AGAMOTTO must retrieve current operational and planning data, such as terrain, mission, and enemy locations, and package this in a standardized format. The simulation and training environment must then either “execute” the mission as planned or simulate a range of potential enemy and friendly actions to compare and optimize COAs. AGAMOTTO must then return the simulation results or optimized COA for further integration into the tactical leaders’ plan.

Figure 1. An abstract representation of the AGAMOTTO design. With further research and development, AGAMOTTO can serve as a simulation and COA-development tool for tactical leaders.
In simulation mode, AGAMOTTO must recreate planned missions, intents, checkpoints, and movement routes to provide the user “playback” of their planned mission, including simulated effects of enemy action and expected weather. This will aid in rehearsal, wargaming, and mission execution. In COA generation mode, AGAMOTTO will ingest current known friendly locations, weather, terrain, unit strengths, and expected missions, then pair these with expected enemy units and locations, and simulate multiple potential enemy and friendly outcomes. This will aid the tactical leader in generating, evaluating, and planning tactical courses of action in real time. These two use cases fundamentally differ in their understanding of the environment, certainty, and visualization. Uncertain outcomes, such as in prediction mode, should be visualized as a range of possible outcomes conditioned on factors such as enemy routes, intent, and strength. In contrast, the wargaming mode assumes relative certainty from a planned friendly operation. It provides the user with a method to visualize their plan, including the effects of terrain and enemy actions.
Challenges
The creation of a system such as AGAMOTTO requires developing several new capabilities. Developing a platform-agnostic method for transferring tactical data will require synthesizing, serializing, and transferring friendly and enemy locations, expected actions, and weather across two disparate C2 and simulation platforms. This will necessitate creating a “wrapper” to retrieve the serialized information from the C2 system, create a synthetic scenario to match it, run the simulation, and return the results to the C2 system for visualization. Additionally, the Army’s C2 systems lack an effective method
for visualizing uncertainty in tactical environments. This is required to convey the uncertainty in the range of potential outcomes generated by AGAMOTTO.
Finally, but not least pressing, AGAMOTTO will require custom-trained agents tailored to arbitrarily defined terrain, enemy, and mission objectives. Unlike previous work, the rules of our “game” will be scenario-specific. RL training in complex environments is challenging under real-time requirements [2]. RL training can take many hours, days, or weeks, as shown in early work optimizing game outcomes [11, 14]. This is not feasible in tactical environments, where planning is often time-constrained to hours. Optimizations are required to reduce these training times and make AGAMOTTO viable. A viable prototype system must overcome these challenges to prove that real-time AI-integrated DSS is viable at the tactical level.
AGAMOTTO Implementation
AGAMOTTO is intended to be a modular system to enable deployment across a variety of NGC2 solutions that could be fielded in the near future. For the tactical planning tool that will serve as the front end for the tactical leader, TAK is a widely fielded visualization, targeting, and planning system in use by the joint force today [17]. TAK also exposes an SDK on some variants of its software, and is a government-off-the-shelf (GOTS) system. TAK mission data (i.e., the mission planning data we seek for our experimentation) can be retrieved and distributed as “cursor on target” messages, a standardized military message format [3]. Using such a format in our implementation means that, as long as the front-end and simulation backend can use this standardized format, the front-end and backend systems will remain platform-agnostic.
RIDE was chosen as the simulation backbone because it is a software package and integrates with Unity RL packages such as MLAgents [10, 21]. This provides both simulation functionality and the ability to harness the simulator’s physics and combat logic for RL agent training. RIDE also provides terrain representations of many areas, including major training centers such as the National Training Center at Fort Irwin, California. This terrain not only has real-world topology, but also a 3D representation of the buildings and other infrastructure. Note that RIDE supports simulation of weather effects, but we do not use that functionality in the initial prototype [22].
To ensure that our prototype can operate on separate networks, a middleware solution is required. Placing a middleware system on a simulation host allows requests and response messages from AGAMOTTO to be sent, provided there is a viable path between the frontend and backend systems, subject to applicable network permissions, firewall rules, and other configuration details.
We chose to begin with a small subset of the many possible missions, terrain, and enemy variations, and to pick a single geographic location for two potential mission tasks. This was done to provide an initial assessment of feasibility and efficacy before scaling to a larger prototype. The location for agent training is the NTC training area, specifically a mockup of a large rural village fictitiously named Razish. A squad-sized unit comprising both friendly and enemy troops, each with about 10 soldiers, is used in our scenario. These soldiers are equipped with small arms and weapons appropriate to their size and threat alignment. No vehicles, artillery, or aircraft are simulated, although RIDE can simulate these capabilities at a small scale in future work.
Initial RL training is performed using the Soft Actor-Critic (SAC) deep reinforcement learning (DRL) method for its general stability, sample efficiency, and built-in exploration incentive [9, 24]. These qualities are expected to assist with efficient training in the very large state space of our lifesized training environment, encouraging model updates for new and unexplored paths, which translates directly to paths in the digital terrain of RIDE. Razish and the NTC are both very large environments in the real and digital worlds, at roughly 1,200 square miles (about the size of Rhode Island). The RIDE MLAgents package provides pre-built SAC functionality, with support for customizing step counts and a variety of other training hyperparameters. Agents are provided with a simple observation space, namely the 3D Cartesian coordinates of their current location and the expected objective location. This objective can be a location or an enemy force, depending on the mission. Finally, to improve training speed, we leverage RIDE’s accelerated scenario timescale options to prevent training from proceeding at the speed of actual ground combat, theoretically making the learning process feasible for tactical use. Figure 2 illustrates both the TAK frontend and the RIDE/Unity backend.


Figure 2. (a) A screenshot from the AGAMOTTO frontend, implemented on TAK. The training area in the image is the Razish village at the NTC at Ft. Irwin, CA.
(b) A screenshot from the AGAMOTTO backend, implemented with RIDE and Unity. The training area in the image is the same village as in 2(a). The screenshot was taken as the friendly squad (center of the screenshot) progressed along its directed route (the white route line in 2(a)) toward its objective (an enemy squad out of view, but depicted as a red diamond on 2(a)).
Fig. 2. A side-by-side view of the functionality of AGAMOTTO. 2(a) and 2(b) show the same geographic area, with 2(a) showing the tactical leader’s view and 2(b) showing the simulation backend.
Finally, the mission-driven reward structure for our agents is a core contribution of this work. Initial attempts to design these values focused on one tactical task and associated purpose: attack to destroy (ATD) [18]. An attack to destroy is an offensive operation that requires the tactical leader to render an enemy force ineffective, typically measured by reducing enemy numbers. This is a simple, straightforward task and purpose that is easily measured to calculate reward.
Positive rewards are given for eliminating enemies in ATD, whereas negative rewards are given for losing friendly troops. Additionally, smaller negative rewards are assigned at each step of the mission’s execution, incentivizing faster solutions. We provide more detail on these rewards in Table I.
| Event | Reward | Notes |
| Enemy unit killed | +5 | Episode success |
| Friendly squad killed | −5 | Episode failure |
| Single enemy killed | +0.45 | Per death |
| Single friendly killed | −0.50 | Per death |
| Closer to enemy | +0.002 | Per decision |
| Farther from enemy | −0.0008 | Per decision |
| Obstacle collision | −0.002 | Per collision |
| Max steps () | 0 | Episode end |
Table I. UNITY EXTRINSIC ENVIRONMENTAL REWARDS FOR ATTACK-TO-DESTROY.
Early Results
Training was conducted on an NVIDIA RTX 5090 GPU and a Dell Alienware Area-51. This is an off-the-shelf system priced at around $4000 to $5000, depending on specific options. We use this system, rather than a more capable compute cluster, to better simulate a locally deployed solution at the tactical level. While cloud compute solutions may be available to the unit, we believe a local GPU with a capable host is a reasonable minimum baseline.
Evaluation of AGAMOTTO COA Generation
We evaluate AGAMOTTO along three axes aligned with NGC2 planning needs: (i) C2–simulation fidelity—whether tactical data encoded as CoT messages reproduce the intended scenario in RIDE; (ii) learning efficiency—whether mission-focused RL training reaches usable policies within tactically relevant wall-clock and step budgets on local hardware; and (iii) mission outcome quality—whether learned behavior improves ATD outcomes relative to ablated training variants. A fourth axis—cognitive and time burden for tactical leaders—is reserved for a planned in-person study.
Evaluation Conditions
While each tactical scenario is different, we must assume a maximum total time allowed to tactical leaders in order to evaluate the feasibility of AGAMOTTO’s design. Additionally, our RIDE does not claim to emulate the movements of live units exactly, defaulting to pre-defined tactical formations (e.g., wedge or file). This oversimplification removes the nuance of scenario-based movement like bounding or covering, but allows for the simulation to be tractable. For our evaluation, we use the conditions below:
Simulation results must be returned in less than 6 hours to be viable. While every scenario is different, at the squad and platoon levels, this is generally the order of magnitude for planning timelines.
We default to tactical movement in fire team wedges aligned in a squad file. There are certainly more nuanced approaches, but we believe this is a good generalization of common movement techniques.
Evaluation Objectives and Research Questions
RQ1 (Integration): Can AGAMOTTO ingest CoT-derived friendly and enemy dispositions (and routes when present) without manual re-entry into the simulator?
RQ2 (Training feasibility): Under fixed hardware (RTX 5090, local deployment per § 5), does SAC training exhibit stable improvement in cumulative return and mission metrics before the maximum step budget and tactical planning time assumptions (e.g., 6-hour maximum for COA generation results generation and feedback).
RQ3 (Reward design): Do mission-task rewards (enemy elimination, friendly loss penalties, step penalties, etc.) measurably affect learned policies compared to ablated reward signals?
AGAMOTTO in Simulation Mode (RQ1)
Initial experiments with AGAMOTTO in simulation mode demonstrated that a single friendly squad can be replicated in RIDE and, when instructed to move along a predetermined avenue of approach (AoA) set by the tactical leader, can advance along it. This was not unexpected, as simulation mode relies solely on mirroring the expected mission and tactical situation in the simulation backend. Using TAK’s route-planning functionality, we assigned a unit to a route, assuming the route’s start is near the intended unit in the digital space. However, it is noteworthy that certain tactical movement techniques are not replicated in RIDE, such as reconnaissance. For example, in a real-world tactical situation, even a unit as small as a squad may send a few soldiers to assess road or terrain conditions or confirm the expected location of the enemy. A real squad may also employ tools such as drones or indirect-fire systems when necessary. AGAMOTTO, in its current RIDE implementation, does not provide a method to simulate these intricate tools. However, simple missions can be simulated, or the tactical leader can use this simulation to estimate movement time requirements and to confirm expected engagement ranges using RIDE’s built-in tactical engagement systems.
AGAMOTTO in COA Generation Mode (RQ 2 and 3)
In a potentially more compelling use case, we now discuss the results of AGAMOTTO’s COA generation mode. This functionality was tested using the same tactical situation as in the simulation tests, with the friendly squad given an ATD task and purpose. After the creation of the applicable units, the friendly unit RL agent can explore paths and actions in accordance with its SAC learning procedures.
It was initially clear that allowing RL agents to physically explore an area as large as NTC, starting with inherently random movement paths, was time-consuming. Early attempts at training convergence exceeded 8 hours and typically ended at the maximum step and episode counts rather than at convergence. With SAC learning occurring in the Unity simulation environment at 50 times the normal speed, the total time to the maximum step count was 3.8 hours, as shown in the first line of Table III. After nearly 4 hours, rewards had converged, but the friendly squad never met the conditions for victory by destroying the enemy unit. To further analyze this shortcoming, we conducted an ablation study of SAC learning variables.
| Condition | Average rewards (last 50% of episodes) | Time (min) | Enemy kills/ep | Friendly losses/ep |
| Baseline SAC | 7.75 | 228.9 | 0.00 | 0.00 |
| No curiosity | 8.20 | 155.2 | 0.00 | 0.00 |
| Lower LR | 1.77 | 209.1 | 0.00 | 1.00 |
| Shorter horizon | 8.66 | 211.6 | 0.00 | 0.00 |
| No squad shaping | 2.04 | 211.0 | 1.00 | 0.50 |
Table III. ATD training ablation summary.
1) Ablation Testing: Table II summarizes the 5 ablation tests, including the baseline SAC parameters. Figure 3 shows the episodic rewards over each ablation and episode. Ablation parameters were chosen to isolate individual SAC parameters and measure their impact on training time and rewards.
Figure 3. AGAMOTTO cumulative rewards for the five ablations.
| Ablation | Intrinsic Rewards | Extrinsic Rewards (SAC) | Unity Environment Rewards | Isolated change |
| Baseline SAC | Curiosity , strength 0.05 | , strength 1.0 | Full ATD (Tab. I) | — |
| No Curiosity | Disabled | , strength 1.0 | Full ATD | No curiosity |
| Lower LR | Same as baseline | , Strength 1.0 | Full ATD | SAC LR |
| Shorter Horizon | Same as baseline | , Strength 1.0 | Full ATD | SAC |
| No Squad Rewards | Same as baseline | , Strength 1.0 | Terminals + distance + collision only | Squad kill/loss 0 |
Table II. Intrinsic and extrinsic reward signals by evaluation ablation.
2) Ablation Testing Results: Table III summarizes ablation outcomes. While the ablation identified methods to marginally increase training rewards, no ablation test succeeded in producing the average enemy-killed-to-friendly-survival ratio that would define success in a tactical ATD mission. Notably, the training times for each of these ablation was inside of our 6-hour evaluation requirements. The obvious convergence shown in Figure 3 indicates that learning occurred, but it failed to find the optimal paths that led to optimal kill/death ratios. This is very likely due to an early convergence of locally optimal paths that do not include the globally optimal paths a tactical leader may create through manual planning. For more information on the effects of this, please see § 6.
AGAMOTTO in Hybrid Simulation/COA Generation Mode
Given these results, we must explore new strategies to theoretically improve RL methods, such as SAC, for tactical COA generation. Generative Adversarial Imitation Learning (GAIL) leverages so-called “expert trajectories” to inform RL agents and streamline training. In practice, GAIL-trained agents can be provided with many examples of near-optimal behavior and then use these examples as a baseline. In our use case, it is a critical insight that we indeed do have a single “expert trajectory”, developed by the tactical leader as part of the planning process. This trajectory is the initial plan, developed by the tactical leader to accomplish their assigned tactical objectives. It is intuitive that this example would be better than a truly random action set, like that of an untrained RL policy, but not optimal in terms of rewards and outcomes.
Given our expert trajectory , a discriminator is trained to distinguish expert state–action pairs from those generated by the policy . The policy is trained to produce trajectories that the discriminator classifies as expert-like.
The GAIL objective is
where is the causal entropy of the policy, encouraging exploration and preventing premature convergence at some local optima. In our use case, this prevents focusing on a single advantageous COA when a different, more optimal COA may still exist.
The discriminator provides a learned reward signal
which is used by a reinforcement learning algorithm to update the policy parameters . At convergence, the policy matches the expert’s occupancy measure and reproduces expert-like behavior without explicitly recovering the expert’s reward function.
It is worth noting that GAIL does not use the environment reward function (e.g., mission-focused rewards) in the same way as our SAC methods do. GAIL trains a policy by imitating the actions performed by the “expert” actor, in this case, those in our tactical leader’s plan. GAIL does not generate actions based on a mission task and purpose, but rather imitates the leader’s actions designed to achieve those goals. This is an indirect way to model an optimal outcome based on those rewards, so we seek a more direct and efficient path to COA generation.
In contrast, advantage-weighted actor-critic (AWAC) combines the expert example with the rewards of the pure-RL example (i.e., our mission-focused reward system) [13]. AWAC assigns high weights to the expert example but does not simply imitate it.
AWAC updates its policy by solving:
where is the critic-based advantage estimate.
The critic is trained via temporal-difference (TD) learning:
This objective performs advantage-weighted behavioral cloning, allowing the policy to imitate high-advantage actions (i.e., our tactical leader’s example) while still improving beyond the demonstrations. AWAC directly uses the mission-focused reward function in its optimization strategy, unlike GAIL.
While currently untested, methods such as AWAC or GAIL may be a way to find tactically optimal outcomes in feasible amounts of time. Issues with early convergence in pure SAC methods show the difficulty of allowing RL agents to learn in massive, highly complex environments such as the Razish training village at NTC.
Discussion
AGAMOTTO is designed to provide tactical leaders with a tool to simulate the outcomes of their plans and a method to generate optimal outcomes given a templated mission and enemy. While the prototype is still under active development, we can report a few interim lessons learned.
Real Time Simulation is Feasible, but Benefits are Still Unknown
AGAMOTTO was initially successful with replicating a TAK-based scenario into Unity using RIDE. While a true in-person evaluation is forthcoming, the methodology outlined in this paper has been shown to be feasible, at least at squad scale, in the initial testing. However, RIDE lacks simulation fidelity that can alter outcomes, such as the ability of combat agents to enter or take cover in buildings or call for indirect fires, if these were planned as part of the operation. This can make the outputs shallow, but not necessarily useless. Existing functionality can depict movement times, engagement ranges, and potential casualty outcomes of a simulated operation. Even without the added fidelity of a more realistic simulation environment, these can assist a tactical leader with planning for operation lead times, casualty evacuation, and firing locations.
Pure RL is likely Insufficient for Real Time COA Generation for Tactical Leaders
We initially attempted to generate novel COAs from a templated friendly and enemy location, given an assigned task and purpose. With a simple ATD task in a highly complex training environment, the agents converged on a locally optimal policy in less than the maximum required time, but likely failed to converge on the globally optimal policy. We believe these issues make SAC-based RL techniques unlikely to be used in tactical settings. To that end, we have explored other RL-based methods that may benefit from the critical insight that we have an expert example in the tactical leader’s planned mission.
Hybrid Imitation/RL Techniques May Provide a Feasible COA Generation Method
AWAC creates optimized action policies using expert examples, such as the planned operation provided by the tactical leader in our use case. While development and evaluation are ongoing, we hypothesize that methods such as AWAC can reduce the time requirements for pure-RL COA generation and provide a viable real-time (or near-real-time) COA generation solution for tactical leaders. If this hypothesis proves valid, we have a solution that a tactical leader can evaluate for efficacy by examining changes in the cognitive and time requirements of planning. Further evaluations can also use the existing RIDE simulation environment to model casualty and mission competition rates, in preparation for a larger in-person evaluation.
Conclusion
Initial development and evaluation of AGAMOTTO show that real-time (or near-real-time) simulation of tactical outcomes is feasible. It also shows that integrating RL for COA generation is challenging and likely requires compromises and the development of new techniques. COA generation techniques that combine pure RL methods with imitation learning show promise for reducing the prohibitive time requirements of pure RL methods, such as SAC. Future work to integrate these methods and conduct a full simulated and in-person evaluation is ongoing.
References
- [1]Jason Adler. Modernizing Military Decision-Making: Integrating AI into Army Planning. en-US. URL: https://www.armyupress.army.mil/Journals/Military-Review/Online-Exclusive/2025-OLE/Modernizing-Military-Decision-Making/ (visited on 03/02/2026).
- [2]Scotty Black. Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning. arXiv:2408.13333 [cs]. Aug. 2024. DOI: 10.48550/arXiv.2408.13333. URL: http://arxiv.org/abs/2408.13333 (visited on 02/27/2026).
- [3]Michael Butler. The Developer’s Guide to Cursor on Target: en. Tech. rep. Fort Belvoir, VA: Defense Technical Information Center, Aug. 2005. DOI: 10.21236/ADA637348. URL: http://www.dtic.mil/docs/citations/ADA637348 (visited on 03/05/2026).DOI
- [4]J. Caballero Toledo, Olaya Pérez-Mon, and María D. R-Moreno. “The role of automated planning in battle management systems for military tactics”. In: Expert Systems with Applications 297 (Feb. 2026), p. 129259. ISSN: 0957-4174. DOI: 10.1016/j.eswa.2025.129259. URL: https://www.sciencedirect.com/science/article/pii/S0957417425028751 (visited on 03/04/2026).
- [5]Army Futures Command. Memorandum - ”Next Generation Command and Control (NGC2) Capability Characteristics of Need” as approved by CSA. May 2024. URL: https://sam.gov/opp/7876c0fdb4dd4520b9e73cc3da1392ba/view.
- [6]Defense Innovation Unit. Thunderforge Project: Integrating Commercial AI-Powered Decision-Making. URL: https://www.diu.mil/latest/dius-thunderforge-project-to-integrate-commercial-ai-powered-decision-making (visited on 03/04/2026).
- [7]Adonisz Dimitriu et al. “A Reinforcement Learning Approach to Military Simulations in Command: Modern Operations”. In: IEEE Access 12 (2024), pp. 77501–77513. ISSN: 2169-3536. DOI: 10.1109/ACCESS.2024.3406148. URL: https://ieeexplore.ieee.org/document/10540085/ (visited on 02/27/2026).DOI
- [8]Vinicius G Goecks et al. “On games and simulators as a platform for development of artificial intelligence for command and control”. en. In: The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 20.4 (Oct. 2023), pp. 495–508. ISSN: 1548-5129, 1557-380X. DOI: 10.1177/15485129221083278. URL: https://journals.sagepub.com/doi/10.1177/15485129221083278 (visited on 02/27/2026).DOI
- [9]Tuomas Haarnoja et al. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv:1801.01290 [cs]. Aug. 2018. DOI: 10.48550/arXiv.1801.01290. URL: http://arxiv.org/abs/1801.01290 (visited on 03/05/2026).
- [10]Arno Hartholt et al. “Introducing RIDE: Lowering the Barrier of Entry to Simulation and Training through the Rapid Integration & Development Environment”. en. In: Proceedings of the Simulation Innovation Workshop (SIW). Simulation Interoperability Standards Organization. Feb. 2021.
- [11]Volodymyr Mnih et al. Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602 [cs]. Dec. 2013. DOI: 10.48550/arXiv.1312.5602. URL: http://arxiv.org/abs/1312.5602 (visited on 02/27/2026).
- [12]Emily Nack and Nathaniel Bastian. “Navigating Challenges and Opportunities in the Cyber Domain With Sim2real Techniques”. In: Cybersecurity and Information Systems Information Analysis Center (CSIAC) 8.2 (2024), pp. 24–39. URL: https://csiac.org/articles/navigating-challenges-and-opportunities-in-the-cyber-domain-with-sim2real-techniques/ (visited on 02/15/2026).
- [13]Ashvin Nair et al. AWAC: Accelerating Online Reinforcement Learning with Offline Datasets. arXiv:2006.09359 [cs]. Apr. 2021. DOI: 10.48550/arXiv.2006.09359. URL: http://arxiv.org/abs/2006.09359 (visited on 03/08/2026).
- [14]OpenAI et al. Dota 2 with Large Scale Deep Reinforcement Learning. arXiv:1912.06680 [cs]. Dec. 2019. DOI: 10.48550/arXiv.1912.06680. URL: http://arxiv.org/abs/1912.06680 (visited on 02/27/2026).
- [15]Kaif Shaikh. Iran war exposes the expanding role of AI in military strike planning. en. URL: https://interestingengineering.com/military/ai-driven-warfare-in-iran (visited on 03/04/2026).
- [16]David Silver et al. “Mastering the game of Go with deep neural networks and tree search”. en. In: Nature 529.7587 (Jan. 2016), pp. 484–489. ISSN: 0028-0836, 1476-4687. DOI: 10.1038/nature16961. URL: https://www.nature.com/articles/nature16961 (visited on 02/18/2026).
- [17]TAK.gov. URL: https://tak.gov/ (visited on 03/05/2026).
- [18]U.S. Army. Army Publishing Directorate. Mar. 2026. URL: https://armypubs.army.mil/ProductMaps/PubForm/Details_Printer.aspx?PUB_ID=1026901 (visited on 03/05/2026).
- [19]U.S. Department of War. Artificial Intelligence Strategy for the Department of War. en. Jan. 12, 2026. URL: https://media.defense.gov/2026/Jan/12/2003855671/-1/-1/0/ARTIFICIAL-INTELLIGENCE-STRATEGY-FOR-THE-DEPARTMENT-OF-WAR.PDF.
- [20]United States, Executive Branch. AI Action Plan. en. URL: https://www.ai.gov/action-plan (visited on 03/04/2026).
- [21]Unity Corporation. What are ML-Agents : ML-Agents Definition. en. URL: https://unity.com/glossary/ml-agents (visited on 03/05/2026).
- [22]USC. Ground Classification Weather. en-US. URL: https://ride.ict.usc.edu/support/documentation/examples/ground-classification-weather/ (visited on 03/05/2026).
- [23]Oriol Vinyals et al. “Grandmaster level in StarCraft II using multi-agent reinforcement learning”. en. In: Nature 575.7782 (Nov. 2019), pp. 350–354. ISSN: 0028-0836, 1476-4687. DOI: 10.1038/s41586-019-1724-z. URL: https://www.nature.com/articles/s41586-019-1724-z (visited on 02/18/2026).
- [24]Ronald J. Williams. “Simple statistical gradient-following algorithms for connectionist reinforcement learning”. en. In: Machine Learning 8.3-4 (May 1992), pp. 229–256. ISSN: 0885-6125, 1573-0565. DOI: 10.1007/BF00992696. URL: http://link.springer.com/10.1007/BF00992696 (visited on 03/05/2026).DOI
- [25]Timothy J. Williams et al. AI-Enabled Wargaming at the U.S. Army Command and General Staff College: Its Implications for PME and Operational Planning. en-US. Jan. 2026. URL: https://smallwarsjournal.com/2026/01/16/ai-enabled-wargaming-cgsc/ (visited on 03/04/2026).
