WorksMathematics
A Transformer-based Approach for Real-time Multi-agent Path Planning with Varying Initial/Final Conditions
Abstract
We address the problem of real-time multi-agent path planning under prescribed initial and final configurations, formulated as a high-dimensional optimal control problem subject to inter-agent collision avoidance and obstacle constraints. Classical solvers based on the Hamilton–Jacobi–Bellman equation or Pontryagin's maximum principle (PMP) become computationally intractable as the number of agents grows, due to the curse of dimensionality and the non-convexity introduced by interaction constraints. We propose OCFormer, a transformer-based neural network that parameterizes the inverse of a coordinate transformation mapping a simple, analytically tractable latent space, in which agents evolve independently under a decoupled Hamiltonian, to the physical space of the coupled, interacting system. The model is trained by minimizing a physics-informed loss comprising the Hamiltonian ODE residual from the PMP and a boundary matching term, without requiring pre-computed trajectory data. The self-attention mechanism endows OCFormer with permutation equivariance over agents, a structural symmetry that prior symplectic network approaches do not natively enforce. When trained on a distribution of boundary conditions, the amortized OCFormer produces near-optimal trajectories for unseen initial and final configurations in a single forward pass (220ms), enabling real-time inference without retraining. We further introduce a lightweight transfer learning framework that adapts a pretrained OCFormer to out-of-distribution perturbations in seconds via L-BFGS fine-tuning. Numerical experiments on two- and three-dimensional multi-agent navigation with up to 128 agents demonstrate the scalability and efficiency of the proposed approach.
Introduction
Motivation and Context
Optimal control problems arise in a wide range of applications, from robotics and autonomous navigation to multi-agent coordination and financial optimization. The goal is to steer a dynamical system from prescribed initial conditions to a desired final state while minimizing a cost functional. Two complementary theoretical frameworks underpin numerical approaches to these problems: Bellman’s dynamic programming principle [3], which characterizes the value function as the unique viscosity solution of the Hamilton–Jacobi–Bellman (HJB) equation, and Pontryagin’s maximum principle (PMP) [1 Tré05], which gives first-order necessary optimality conditions through the introduction of a co-state variable. Despite their theoretical power, both frameworks face severe computational challenges as the state dimension grows. Grid-based methods for the HJB equation suffer from the curse of dimensionality [3]: their complexity scales exponentially with the number of state variables, making them infeasible beyond a few dimensions. Indirect shooting methods based on the PMP avoid explicit gridding of the state space but require repeated numerical integration of the Hamiltonian system and are highly sensitive to the initial guess of the co-state, often failing to converge without a careful initialization [Tré05]. In multi-agent path planning, where agents with individual state dimension yield a joint state of dimension , these difficulties are compounded by the need to enforce inter-agent collision avoidance constraints that introduce non-convexity and long-range coupling among the agents, rendering classical solvers impractical for real-time use at scale.
To overcome these scalability limits, deep learning has motivated mesh-free methods that parametrize solutions directly with neural networks, avoiding explicit state-space discretization. Building on early residual-based parametrizations of differential equations [15], high-dimensional extensions such as the Deep Ritz method [8] and backward-stochastic-differential-equation reformulations [9] scaled these ideas to hundreds of dimensions, and physics-informed neural networks (PINNs) [RPK19] unified them by encoding governing equations and boundary conditions as soft loss constraints, a paradigm that has since grown into the broader field of scientific machine learning [ZSX+22, ZKKS24, TZK+24, ZWK+25, ZKK25, KZGK26, WZK26, ZK26]. In the optimal control setting, the PMP’s Hamiltonian ODE system provides a natural PINN residual; SympOCNet [MZDK22] pioneered this for multi-agent path planning by pairing a symplectic architecture [11] with PINN training, later extended to time-dependent and non-convex settings [ZWL+24]. However, these architectures process agents in a fixed sequential order and impose a possibly-unnecessary symplectic constraint. Neither property reflects the permutation symmetry inherent to multi-agent problems.
Beyond the symplectic-PINN line of work, several alternative paradigms merit mention: optimal-transport-based continuous normalizing flows for mean-field control [19], adaptive deep learning for the HJB equation [NZGK21], and Hopf–Lax representation formulas that solve Hamilton–Jacobi equations without spatial gridding [7, 13]. In the multi-robot community, graph neural networks [14] target decentralized collision avoidance with a similar permutation-equivariance motivation but distributed rather than centralized optimality. Classical planners such as ORCA [VDBGLM11], conflict-based search [SSFS15], and sequential convex programming [2] remain practical workhorses that trade global optimality for tractability.
The transformer architecture [VSP+17] directly addresses this gap. Its self-attention mechanism computes pairwise interactions among all sequence elements simultaneously, capturing long-range dependencies and permutation symmetries that convolutional or recurrent architectures handle only approximately, and it is theoretically a universal approximator of continuous, permutation-equivariant sequence-to-sequence functions [YBR+19], consistent with its empirical dominance across NLP [6, 4] and vision [DBK+21]. This permutation-equivariant structure is especially relevant for multi-agent problems, where agents are interchangeable and a solution method should be invariant to their indexing, allowing a transformer to learn collective, collision-avoidance behaviors that generalize across agent configurations without a fixed ordering.
Contribution
Contribution. In this work we propose OCFormer, a transformer-based framework that addresses multi-agent optimal control by combining the structural inductive biases of attention with physics-informed training. The central idea is to learn a parameterized inverse coordinate transformation that maps analytically tractable latent trajectories, corresponding to non-interacting agents with a decoupled Hamiltonian, to the physical trajectories of the interacting system. OCFormer is built from alternating transformer blocks that mirror the leapfrog structure of Störmer–Verlet integration [11], and is trained by minimizing a physics-informed loss comprising the Hamiltonian ODE residual from the PMP together with a boundary matching term enforcing prescribed initial and final configurations. Unlike SympOCNet [MZDK22], OCFormer is not constrained to be symplectic; instead, it achieves permutation equivariance through the self-attention mechanism, which is essential for generalization over varying agent counts and configurations. Once trained on a distribution of boundary conditions, OCFormer infers near-optimal trajectories for unseen initial and final configurations in a single forward pass ( ms), enabling real-time multi-agent path planning. We further develop a transfer learning strategy that efficiently adapts a pretrained OCFormer to perturbed boundary conditions, avoiding costly retraining from scratch. The framework is validated on two- and three-dimensional multi-agent navigation benchmarks, demonstrating competitive trajectory quality and significantly reduced inference time compared to existing methods.
Multi-Agent Optimal Control Problem
In this section, we consider a class of multi-agent path planning problems involving collision and obstacle avoidance, with fixed initial and final constraints, and derive the associated Hamiltonian boundary-value ODE system. This is a concrete instance of the general optimal control problem and Pontryagin’s maximum principle (PMP) reviewed in Appendix § 7. We present a transformer based learning method for approximating the solution together with the optimal trajectories of these problems.
Problem setup
Problem setup. We are interested in a class of multi-agent path planning problems. A similar problem has been explored by the authors in [ZWL+24]. Here, we give a brief overview of the setup. We consider a system consists of elementary sub-systems in interaction, and evolves in time horizon to . The state and the control of each sub-system have dimension and , respectively, leading to a concatenated state and a concatenated control , for every . Each sub-system has its individual potential energy functional, , associated with the state , and individual kinetic energy functional, , associated with control . Moreover, we assume that the individual kinetic energy functional is convex w.r.t . Each sub-system is governed by a dynamical system of the form
i.e., the individual dynamical system is linear w.r.t control . The interaction (collision and obstacle avoidance) between the sub-systems will be represented by , that imposes a constraint on the state variable such that
This constraint can be converted to the Lagrangian of the system (see details in [MZDK22], and a variant in [ZWL+24]), by introducing a soft penalty function , defined by
Figure 1. Flowchart of the OCFormer framework. For a pair of prescribed initial and terminal configurations, boundary data are first encoded into latent variables , which serve as the input to the transformer-based inverse coordinate map . The network predicts the physical position and momentum trajectories over the time grid without solving an ODE at inference time. During training, the predicted trajectories are differentiated with respect to time and evaluated in the Hamiltonian system so that the model satisfies the physics-informed residual loss associated with Pontryagin’s maximum principle, while a boundary matching term enforces the initial and final constraints. The total loss is minimized over the network parameters, yielding a model that directly maps varying boundary conditions to feasible multi-agent trajectories in real time.
where for every , is defined by
The two branches of (3b) are designed to meet at with continuity: both the value and the derivative match, ensuring smooth gradients during optimization. The max over constraints in (3a) penalizes only the most violated constraint at each collocation point, which makes the barrier numerically tractable with small (we use , throughout). We note two consequences: (i) the log-barrier is not Lipschitz continuous at the constraint boundary, so the standing regularity assumptions on the Lagrangian in Appendix § 7 must be understood to hold on the interior of the feasible set; (ii) the -penalized solution is biased away from the exact constrained optimum, and the values of and control the cost–feasibility trade-off. In this way, the reduced Lagrangian of the system is given by the sum of the potential and kinetic energies of the individual sub-systems, together with the penalty term (3). Moreover, the initial and final conditions of the system are constrained to two sets, and , which are compact convex subsets of , i.e.,
From Maximum Principle to Hamiltonian ODE system
In what follows, we denote
We seek a trajectory , that minimizes the total action functional of the system, while satisfying the initial/final constraints (5) and state constraints (2). By employing the soft penalty method as in (3) for the state constraints, this problem can be reformulated within the optimal control framework introduced in (24) (Appendix § 7), by setting
for every in (24a) and (24b), and be the identity map with
Let us denote
where denotes the Legendre-Fenchel transformation of , and we use the property that is linear w.r.t . For sake of presentation, we consider are two point sets respectively, and the extension to general sets is straight-forward. In that case, following the maximum principle (25) (Appendix § 7), we have that the optimal trajectory together with the co-state satisfy the following condition: for every ,
with respect to the initial, final conditions
and is taking the value such that the maximum in (7) is obtained. Notice that, since the initial and final conditions are two points, and the terminal cost is zero, there are no constraints on and . Our focus in the remainder of this paper is to solve the system (7) associated with the multi-agent path planning optimal control problems.
The OCFormer Framework
Trajectory Representation by Coordinate Transformation in Latent Space
In the context of multi-agent path planning problems, the individual potential energy function and the individual kinetic energy function are typically well-behaved convex functions, often of quadratic form. Moreover, the state and control dimensions and of the sub-systems are generally small, typically equal to 2 or 3 in practical physical scenarios. Consequently, (one of) the principal challenges arises from the interactions among the sub-systems, represented by in the system’s Lagrangian and converted to the Lagrangian through the soft penalty term , which effectively serves as an “interaction” potential energy. These interactions render the overall system non-convex in the dimension .
The key idea of our method is employing a change of variable operator, , in the physical space, transforming the trajectory functional satisfying the (PMP) system (7) of the original interaction system, to a “simpler” one (see Figure 2), that is,
Figure 2. Coordinate-transformation view of the proposed trajectory representation. The forward map sends the physical trajectory pair to a simpler latent trajectory pair associated with a decoupled Hamiltonian system, while the inverse map reconstructs the interacting multi-agent trajectories in physical space.
This trajectory is simpler in the sense that it corresponds to the solution of a PMP system consisting of non-interacting sub-systems, i.e., a system having same time horizon, state/control space, dynamics, initial/final constraints as the original one, but with Lagrangian taking the form
Thanks to the convexity and low dimensionality, the system (7) with Lagrangian (9) can be easily solved, by solving the low dimensional sub-systems separately. We call the trajectory functional in latent space. To be more precise, the two maps , are the solutions of the following system: for every
where we match the initial, final conditions of to , that is
and the Hamiltonian is defined by
Parameterize the Inverse Operator by Transformer Based Neural Network
Assumption 1. There exists a diffeomorphism such that the latent trajectory satisfies the decoupled Hamiltonian system (10) with Lagrangian (11). Although the existence of such a map is not guaranteed in full generality and requires further study, this assumption is the working hypothesis that motivates the neural parameterization of .
Remark 1 (Straight-line latent trajectories and the residual inductive bias). For the specific problems considered in this work, the dynamics are and the individual kinetic energy is quadratic, so . Moreover, , , and the initial/final constraints fix , as singletons. The latent system (10) then reduces to a constant-velocity straight line:
Consequently, the ansatz (18) collapses to a neural parameterization of a curve in ,
where denotes the learned correction (the non-identity part of ). The residual structure means the ansatz is initialized exactly at the non-interacting optimal solution, the straight-line, constant-velocity paths that solve the unconstrained problem, and learns the collision-avoiding correction on top. This is a strong inductive bias: the network only needs to model the deviation from the trivial solution. We also note that in the per-instance training regime, the network only ever sees a one-dimensional line segment of its input domain (the set ), which explains both why the fit is easy and why it does not directly generalize to new boundary conditions.
In this section, we approximate and parameterize the inverse of the operator by , where represents the unknown parameters to be learned through a transformer-based architecture.
Definition 1. The OCFormer is defined as the following neural network:
where is the identity mapping, are tokenwise deembedding MLPs, is the tokenwise embedding layer, are the transformer blocks defined below in terms of the standard self-attention operation [VSP+17]:
for every , where is the embedding dimension. We use for numerical examples.
To accommodate the sequence-to-sequence nature, the inputs and are first reshaped into (treating each of the agents as a token of dimension ) and the outputs are reshaped back into . The details of the OCFormer structure are illustrated in Figure 3.
Figure 3. Architecture of OCFormer. are first reshaped into . are tokenwise embedding layers, which are applied on the last axis of reshaped input. . Partial transformer blocks and are depicted by the dashed rounded rectangle. is identity mapping in . () are the transformer blocks. are tokenwise deembedding MLPs, whose output are flattened back into . is the identity mapping in , serving as residual connection.
Remark 2 (Symplecticity, permutation equivariance, and limitations). Unlike SympOCNet[MZDK22], OCFormer is not constrained to be symplectic and is only invertible in the latent space. As shown in Proposition 1, the PINN loss directly enforces the Hamiltonian dynamics, making a structural symplectic guarantee unnecessary.
OCFormer is permutation equivariant over agents: any permutation of the agent indices applied to the input produces the same permutation of the output. This follows directly from the architecture: the tokenwise embeddings and deembedding MLPs are applied independently to each agent via vmap, the self-attention mechanism in is permutation-equivariant by construction, and the residual connection preserves equivariance. Formally, if denotes the permutation matrix corresponding to , then .
A notable failure mode arises from this design: with no positional encoding, two agents that share identical latent inputs produce identical outputs and can never be separated. This symmetry-breaking limitation is particularly relevant for the symmetric benchmark in § 5, where symmetry breaking must be driven by random weight initialization alone.
For problems where agents are not interchangeable (e.g. the 3D room with role-differentiated targets in § 5), we use PositionalOCFormer, a variant with per-agent weight matrices that deliberately breaks permutation equivariance.
The OCFormer Method and Training procedure
Let us recall that the state and co-state functionals are obtained by applying the inverse of the coordinate transformation operator , to the latent state and co-state functionals . Approximating this inverse of transformation by , one obtain an approximation of the state and co-state functionals
which depends on the learnable parameters . Then, one can compute the derivative of w.r.t by
where denotes the Jacobian matrix. In order to learn the parameters , we construct and minimize a physics-informed loss function of . In particular, a set of collocation points is randomly sampled in . Then, together with can be computed by numerically solving the latent system (10). The physics-informed loss function is then defined by
where
and
where denotes the first components of . For 3D experiments we additionally include a velocity bound penalty
where is the finite-difference velocity computed from the predicted trajectory and , which enforces a soft box constraint on the control magnitude. For the ball-3D problem we further add an explicit obstacle penetration penalty
where is the signed squared-distance constraint for the spherical obstacle; this term directly penalizes penetration at collocation points, supplementing the log-barrier inside .
Proposition 1 (Zero-loss implies extremality). If on a dense set of collocation points and , then, assuming the neural ansatz is continuously differentiable, the trajectory satisfies the Hamiltonian ODE system (7) on and is therefore a normal extremal of the penalized problem. In practice, the loss is driven
to a small but nonzero value; the proposition guarantees that driving it toward zero yields an increasingly accurate approximation of an extremal. This holds regardless of whether is symplectic, so the symplectic constraint of prior work [MZDK22] can be dropped.
A full pseudocode listing of the training procedure sketched above is given in Algorithm 1, Appendix § 9.
A Real Time Learning Framework for Perturbation on initial and Terminal Conditions
Perturbation on Initial and Terminal Conditions
4.1. Perturbation on Initial and Terminal Conditions. We introduce the following optimal control problem with perturbation on initial and terminal conditions:
where are perturbations on the initial and terminal conditions. If the perturbation is structurally simple (e.g., a rigid body transformation) or sufficiently small, solving the entire optimal control problem from scratch is computationally redundant, as the complex multi-agent collision avoidance logic remains largely consistent. To enable real-time path planning under varying conditions, we propose a transfer learning framework that exploits a pre-trained reference model.
Transfer Learning Framework
4.2. Transfer Learning Framework. Suppose we have a base transformer model, , already trained on a reference setup (unperturbed problem). We freeze its parameters and introduce lightweight, trainable coordinate transformation modules before and after the base model:
where is a trainable global affine transformation and is a trainable per-agent bounded correction, defined below. Depending on the type of perturbation, one or both modules are activated.
Affine module . and act on the phase space coordinates as , with a learnable matrix applied to both position and momentum, and a learnable translation vector applied only to position. The residual form initializes at the identity at , . We note that is a general linear transformation, not constrained to , and and are trained independently without enforcing ; in practice, the two modules learn approximately inverse transformations from the data. Because the total parameter count is (e.g., 8 parameters for ), the network adapts rapidly via L-BFGS, typically converging in under 100 iterations.
Correction module . and apply per-agent, per-dimension corrections, parameterized by learnable parameters (one scalar per agent per spatial dimension), acting as , where the sine nonlinearity bounds each correction component to . The parameter (set to 0.04 in our experiments, giving ) controls the maximum correction magnitude; it is chosen independently of the perturbation magnitude and provides a conservative envelope that accommodates the required trajectory adjustments. and are time-independent parameters. The time-varying correction to the trajectory is generated by perturbing the latent input , which then propagates nonlinearly through the frozen base model . The forward correction adds a residual adjustment at the output. Both modules together contain trainable parameters, which is negligible compared to the frozen base model.
Two operating regimes. When the initial and terminal conditions are transformed by an element of the or group (i.e., a global rotation and translation), the global affine module already provides the correct symmetry class, so we set (equivalently ) and train only . When boundary conditions are instead subjected to random, unstructured noise, a global affine transformation alone may not provide sufficient degrees of freedom to satisfy all local constraints, so we additionally activate to supply the missing per-agent corrections. In both regimes, the new transfer parameters are optimized using the original physics-informed loss function (16). Training is performed on boundary data within a room of size , which is twice the linear dimension of the room used for the base model. This enlarged transfer domain ensures that the adapted trajectories remain well within the feasible region during optimization. The lightweight parameterization accelerates convergence compared to training from scratch, typically requiring only seconds of L-BFGS fine-tuning.
Numerical Experiments
Optimal Path Planning for Multiple Drones in a 2D Room
In this section, we consider the path planning problem for multiple drones in a 2D room. The drones are expected to reach the target positions at the same time without collision. The room is a square . The radius of the drones are . The time horizon is . The problem can be formulated as
Symmetry Setup. We apply the base OCFormer (permutation-equivariant) to drones. For , all drones are placed on a single circle of radius . For , drones are distributed across two concentric circles (16 agents each, radii and ). For , two concentric circles with 32 agents each (radii , ) are used, with drone radius . For , four concentric circles with 32 agents each (radii ) are used with . In all cases targets are the antipodal points of the initial positions.
The optimality gap increases with agent density but remains under 10% for all cases, and for the OCFormer cost is within 1% of the true optimum obtained via pseudospectral direct transcription (NLP ground truth). A seed-sensitivity study across 5 random seeds for yields cost (mean std), confirming stability. For , OCFormer and SympOCNet achieve near-identical costs, confirming that OCFormer matches the state of the art on small problems. At , OCFormer reduces the cost by 13% relative to SympOCNet, and at , OCFormer converges to a feasible solution while SympOCNet fails to converge. We attribute this advantage to the permutation-equivariant attention mechanism, which shares parameters across agents and provides a better-conditioned optimization landscape at larger , compared to SympOCNet’s fixed-ordering gradient modules.
Figure 4. Predicted trajectories at for drones, the largest and most demanding configuration in Table 1. Drones shown as colored circles with paths as gray curves.
Table 1. Quantitative results for the 2D room symmetry setup, and direct comparison against SympOCNet (JAX implementation) on the identical circle benchmarks, using the same boundary data, parameters (, , room size, , , ), and training budget (100k–500k Adam + L-BFGS on an NVIDIA RTX A6000). is the analytic lower bound (unconstrained optimum). Cost is evaluated via trapezoidal integration of ; Gap is relative to . SympOCNet was not evaluated at . For every configuration, OCFormer’s fine-grid constraint minimum is positive, in the narrow range , confirming collision-free trajectories throughout.
| Layout | OCFormer | Gap | SympOCNet | Feasible? | |||
| 4 | 0.02 | 0.200 | 0.2003 | 0.2% | 0.2001 | Yes | |
| 8 | 0.02 | 0.400 | 0.4036 | 0.9% | 0.4046 | Yes | |
| 16 | 0.02 | 0.800 | 0.8307 | 3.8% | 0.8345 | Yes | |
| 32 | 0.02 | 1.156 | 1.2531 | 8.4% | 1.4413 | Yes | |
| 64 | 0.01 | 2.297 | 2.448 | 6.6% | — | — | |
| 128 | 0.01 | 4.704 | 5.1424 | 9.3% | 13.32 | No |
5.1.2. Symmetry Setup with Rigid Body Transformation. The running cost , the pairwise distance constraints, and the free dynamics are all invariant under simultaneous rotation and translation of all agent coordinates (the group). Consequently, if is the optimal trajectory for the reference boundary conditions, then is the exact optimal solution for the transformed problem (28). We use this analytically tractable setting as a verification experiment: the rigid-body transfer method should recover the known exact solution without error. The problem can be formulated as
where
are the rotation matrix and translation vector respectively, are the initial and terminal positions of the drones in the reference symmetric setup. We apply the rigid-body transfer method to the cases with drones, where the base OCFormer is trained with the reference setup. The affine module uses a general matrix (not constrained to ) and the forward and inverse modules are trained independently, providing additional flexibility at the cost of not strictly enforcing . We report the trajectory error against the exact solution . The learned affine parameters recover the exact transformation to within numerical precision (trajectory error ), confirming that the transfer framework correctly handles the -invariant case.
Random Perturbation on Initial Conditions
We perturb the initial conditions by adding a uniformly distributed random noise to the initial positions of the drones. The perturbation magnitude is and the target positions are the antipodal points of the unperturbed initial positions. Unlike the case, the resulting boundary conditions are not related by any global symmetry transformation, so no closed-form solution exists; the perturbations are local and unstructured, so the new problem cannot be matched by a single global rotation or translation, which motivates the addition of bounded correction layers on top of the pre-trained base model. The perturbed initial and prescribed terminal configurations are shown in panels (a) and (d) of Figure 5 below. We apply the random-perturbation transfer method to the cases with drones. The results are depicted in Figure 5.
Figure 5. Predicted multi-agent trajectories for the random-perturbation experiment with drones at times , , , from left to right. Starting from the perturbed initial condition, the transferred model generates collision-free paths that progressively reorganize the swarm toward the prescribed terminal arrangement. The intermediate snapshots show that the learned corrections adapt the reference solution while preserving the overall interaction structure encoded by the original OCFormer.
Across 100 independent random-perturbation trials, fine-tuning consistently reaches a low residual loss (–), the trajectory cost concentrates around 1.26, and the constraint minimum remains strictly positive throughout (–), confirming that the transfer learning framework reliably produces feasible, near-optimal solutions under random unstructured perturbations.
Amortized OCFormer: Training on a Distribution of Boundary Conditions
The per-instance training in §5.1.1 solves one pair of boundary conditions at a time. We now demonstrate that OCFormer can be amortized: trained on a distribution of boundary conditions and evaluated zero-shot on unseen pairs, without any architectural change. The key observation is that the latent pair is a bijective encoding of : given boundary conditions, the straight-line latent trajectory is computed analytically and fed to the same OCFormer architecture. Amortization therefore requires only a batched training loop: at each iteration we sample boundary pairs, build the straight-line latent for each, and average the PINN and boundary losses over the batch.
We train on 100 boundary pairs generated by applying random 2D rotations (angles uniformly drawn from ) to the and antipodal circle configurations, and test zero-shot on 20 held-out rotation angles. Training uses the same Adam + L-BFGS pipeline as Algorithm 1, with batch size . Results are reported in Table 2.
| Test cost (mean std) | Gap vs. | Feasibility | Inference (ms) | |
| 8 | 1.3% | 100% | ||
| 16 | 5.0% | 75% |
Table 2. Zero-shot amortized inference results. Trained on 100 rotated boundary pairs, tested on 20 held-out rotations. Inference time is the wallclock time for a single forward pass (JIT-compiled, averaged over 100 runs).
For , the amortized model achieves near-perfect zero-shot generalization: trajectories for all 20 held-out boundary pairs are collision-free, and the cost is within 1.3% of the analytic lower bound. For , the cost gap increases to 5.0% and feasibility drops to 75%, indicating that the denser agent configuration pushes the limits of the current training setup. Inference requires a single forward pass through OCFormer, taking approximately 220 ms on an NVIDIA RTX A6000 regardless of . This is three orders of magnitude faster than per-instance training ( minutes for Adam + L-BFGS) and directly substantiates the “real-time” label for in-distribution boundary conditions.
Path Planning for Multiple Drones in a 3D Room
In this section, we consider the path planning problem for multiple drones in a 3D room. The drones are expected to reach the target positions at the same time without collision. The room is a box with two internal axis-aligned barrier obstacles (positioned as specified in the code release). The radius of the drones is and the time horizon is . Unlike the symmetric 2D circle setup, the 3D room has fixed geometry with role-differentiated start and target positions, so the agents are not interchangeable under permutation. We therefore use PositionalOCFormer, a variant of OCFormer in which the tokenwise embeddings and decoders carry per-agent weights (via PositionalLinear layers with parameters ), explicitly breaking permutation equivariance in favor of agent-specific representations. The problem can be formulated as
where the box constraint is enforced through the penalty function together with the inter-agent collision constraint. We apply the PositionalOCFormer to the cases with drones. The results are depicted in Figure 6.
Figure 6. Predicted trajectories for the 3D room experiment with drones at times , , , from left to right. The snapshots illustrate how the learned model coordinates a large collection of agents in a confined three-dimensional domain, moving them from their initial configuration to the terminal one while maintaining collision-free motion throughout the room. The intermediate frames highlight the scalability of OCFormer to high-dimensional multi-agent path-planning problems.
Ball 3D Example
In this section, we consider the path planning problem for multiple agents navigating around a spherical obstacle in 3D space. The agents are expected to reach the target positions at the same time without collision and while avoiding the ball-shaped obstacle. The problem can be formulated as
where is the center of the spherical obstacle and is its radius, with time horizon . We apply OCFormer to the cases with and drones. The results of drones are depicted in Figure 7.
Figure 7. Predicted trajectories for the 3D ball-obstacle experiment at times , , , from left to right. The figure shows the agents moving from their initial to terminal positions while simultaneously avoiding pairwise collisions and steering around the spherical obstacle. The intermediate configurations make clear that the learned dynamics capture both inter-agent avoidance and obstacle-avoidance behavior in a genuinely three-dimensional setting.
Conclusion and Future Work
We have presented OCFormer, a transformer-based neural network for multi-agent optimal control that parameterizes the inverse of a coordinate transformation from a decoupled latent space to the physical interacting system. Trained with a physics-informed loss combining Hamiltonian ODE residuals and boundary matching, OCFormer solves the two-point boundary value problem for fixed initial and final configurations without requiring pre-computed trajectory data. A lightweight transfer-learning framework adapts a pretrained model to rigid-body transformations and random perturbations of the boundary conditions, requiring only seconds of L-BFGS fine-tuning rather than full retraining.
Our numerical experiments demonstrate that OCFormer produces collision-free trajectories on 2D and 3D multi-agent benchmarks with up to 128 agents, achieving costs within a few percent of rigorous lower bounds. The permutation-equivariant architecture provides a useful inductive bias for symmetric problems, while its positional variant handles asymmetric settings.
The amortized OCFormer demonstrated in § 5.3 confirms that zero-shot generalization is achievable with no architectural change, but the current training is limited to rotated versions of a fixed configuration. Scaling amortized training to broader distributions (e.g., arbitrary start/goal pairs, varying agent counts) and improving feasibility at higher densities are natural next steps. Other directions include development of a symplecticity-preserving OCFormer variant, extension to general nonlinear dynamics beyond the current setting and integration with feedback control for online replanning.
General Pontryagin's Maximum Principle with State Constraints
We are interested in the finite horizon deterministic optimal control problem as follows:
over satisfying
for all , with a particular initial/final constraint
Here, is the time horizon, is the state set and is the control set. is the space of admissible controls. The running cost , the dynamics are given functions, with basic regular properties: continuous, Lipschitz continuous w.r.t all variables. Moreover, the initial and final conditions are subject to a particular constraint (34), where and is a nonempty closed convex subset of . We also assume basic regular properties on : with Lipschitz derivatives. Then, the system (33), subject to (34), admits a unique, continuous solution for every (see for instance [Tré05]), and we denote this solution.
Let us denote the state space of the control problem (24). We call a trajectory if . A necessary optimality condition for the control problem (24) is given by the Pontryagin's maximum principle (PMP), by introducing the co-state in the topological dual space of , the cost multiplier and the initial/final constraints multiplier . Here, for simplicity, we assume further regularity property that is . Then, any optimal trajectory satisfies the following system (see for instance [1])
with the pre-Hamiltonian defined by
Moreover, the Pontryagin multiplier satisfies
with the initial/final Lagrangian defined by
as well as
and the following Hamiltonian inequality holds: for every
In (39), denotes the normal cone to at . Recall that for any closed convex subset of a Banach space , the normal cone to at is defined by
We note that pure state constraints , as used in the main text (2), generally induce measure-valued multipliers in the PMP framework [10, 16]. The standard (absolutely continuous) PMP presented above does not directly cover state constraints; the soft penalty reformulation (3) used throughout the main text is what allows us to apply it.
Classical Indirect Shooting Method
Up to rare cases, the system (25) can only be solved approximately using numerical methods. Despite its fairly comprehensive theoretical characterization, its numerical approximation remains a challenging task. One classical numerical approach is the shooting method. For simplicity, we assume the data of the control problem (24) are with Lipschitz second order derivatives, and we consider equality initial/final constraints . Consider the shooting parameter , and the shooting function , defined by
where is the solution of system (25) with the initial/final condition replaced by the initial condition , and . Notice that there are an equal number of equations and unknown variables in (42). Then, is a regular zero of . Therefore, we convert the initial-final system (25) into a root finding problem, which is typically solved by a (quasi-)Newton method. However, one can find that this method is highly sensitive to the initial guess of shooting parameter, which may cause it to fail to converge. This ill-conditioning is precisely the practical motivation for the learning-based approach developed in the main text, which avoids repeated numerical integration and root-finding altogether.
Training Algorithm
Algorithm 1 gives a full pseudocode listing of the training procedure sketched in § 3.3, for a fixed iterations.
Algorithm 1 OCFormer Training Procedure
Require: Boundary conditions ; Hamiltonian with soft-penalty parameters ; boundary weight ; Adam iterations ; L-BFGS iterations ; optional velocity bound (3D only); optional auxiliary penalty weight (ball-3D only).
Ensure: Learned physical trajectories .
Pre-solve the non-interacting latent system (10) analytically: for each agent , compute (constant) and .
Initialize the trainable parameters of OCFormer.
for to do
Sample collocation points uniformly at random in .
Evaluate the latent trajectories analytically at using the precomputed closed-form expressions from Step 1.
Compute physical trajectories and time derivatives via (14)–(15) using the Jacobian-vector product (JVP) through OCFormer.
Compute total loss from (16).
Update with one Adam gradient step.
end for
Fine-tune for iterations using L-BFGS on a fixed uniform grid with the same loss.
return Learned trajectories .
References
- [1][AS13] Andrei A. Agrachev and Yuri Sachkov. Control theory from the geometric viewpoint, volume 87. Springer Science & Business Media, 2013.
- [2][ASD12] Federico Augugliaro, Angela P Schoellig, and Raffaello D’Andrea. Generation of collision-free trajectories for a quadrocopter fleet: A sequential convex programming approach. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1917–1922. IEEE, 2012.DOI
- [3]Richard Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.pubmed.ncbi.nlm.nih.gov/17730601
- [4]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020.arxiv.org/abs/2005.14165
- [6]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186. Association for Computational Linguistics, 2019.
- [7]Jérôme Darbon and Stanley Osher. Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Research in the Mathematical Sciences, 3(1):19, 2016.
- [8]Weinan E and Bing Yu. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.arxiv.org/abs/1710.00211
- [9]Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
- [10]Richard F Hartl, Suresh P Sethi, and Raymond G Vickson. A survey of the maximum principles for optimal control problems with state constraints. SIAM Review, 37(2):181–218, 1995.
- [11]Pengzhan Jin, Zhen Zhang, Aiqing Zhu, Yifa Tang, and George Em Karniadakis. Sympnets: Intrinsic structure-preserving symplectic networks for identifying hamiltonian systems. Neural Networks, 132:166–179, 2020.
- [13]Alex Tong Lin, Yat Tin Chow, and Stanley Osher. A splitting method for overcoming the curse of dimensionality in Hamilton–Jacobi equations arising from nonlinear optimal control and differential games with applications to trajectory generation. Communications in Mathematical Sciences, 16(7):1933–1973, 2018.
- [14]Qingbiao Li, Fernando Gama, Alejandro Ribeiro, and Amanda Prorok. Graph neural networks for decentralized multi-robot path planning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11785–11792. IEEE, 2020.
- [15]Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987–1000, 1998.
- [16]Helmut Maurer. On optimal control problems with bounded state variables and control appearing linearly. SIAM Journal on Control and Optimization, 15(3):345–362, 1977.
- [19]Derek Onken, Samy Wu Fung, Xingjian Li, and Lars Ruthotto. Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9223–9232, 2021.






