Language-Guided Terrain-Adaptive Neural MPC for Autonomous Traversal of Articulated Tracked Robots

Introduction

Recent vision-language-action (VLA) models show that large-scale visual and linguistic priors can be transferred to robotic control by mapping observations and language instructions directly to actions [1], [2]. This paradigm is promising for semantic generalization, but it is not sufficient by itself for contact-rich autonomy. In many deployments, the action generator is executed as a feedforward policy conditioned on the current observation; it does not explicitly optimize over future contact events, actuator limits, body-attitude bounds, or task-specific safety constraints. Consequently, when the robot encounters a new terrain geometry or a changed traversal preference, the observation can become out of distribution and the policy may produce actions that are semantically plausible but dynamically infeasible.

This limitation is especially pronounced in urban search and rescue, where articulated tracked robots (ATRs) must traverse stairwells, landings, and cluttered building interiors [3], [4], [5]. Their articulated flippers enlarge the contact patch, improve traction, and facilitate obstacle traversal [6], [7] (see Figure 1), but reliable motion requires coordinated chassis–flipper behavior under hybrid robot–terrain interaction (RTI). Frequent support-mode switches and discontinuous contacts make explicit modeling difficult, while small geometric variations can change the desired trade-off between stability, impact, and speed. These factors motivate a controller that

can retain hard feasibility constraints while adapting its objective to different environments and operator preferences. Classical geometry-based planners ensure basic traversability but rely on pre-defined robot configurations, which limits optimality and often produces non-smooth trajectories on irregular terrain [8], [9], [10]. Reinforcement learning avoids explicit modeling yet commonly exhibits jerky switching in discrete action spaces, oscillations in continuous action spaces, and requires terrain-specific reward shaping [11], [12], [13], [14], [15], [16], [17]. Vision-language policies improve semantic grounding, but direct action prediction still lacks an explicit mechanism for enforcing traversal constraints and retuning objective trade-offs under out-of-distribution terrain.

Model predictive control (MPC) offers a complementary route: it enforces hard constraints, coordinates multiple objectives via tunable cost weights, and yields smooth behavior through receding-horizon optimization. However, practical MPC deployment on ATRs faces two obstacles. First, accurate modeling under hybrid contacts is difficult; non-smooth contact transitions and support changes make real-time numerical optimization challenging. Second, tuning is labor-intensive: weights and constraints in a multi-objective formulation often require scenario-specific adjustment by experts. To address these challenges, we present ASTRIL-MPC, a language-guided neural kinematics MPC framework for ATRs. Prior work has shown that neural NMPC can be deployed in safety-critical domains such as automated driving under unknown friction conditions [18]. We augment MPC with a learned kinematic predictor to bypass explicit contact modeling, and introduce a regulation layer powered by a large language model (LLM). Rather than generating actions directly, the LLM acts as a high-level optimizer over a restricted parameter set, translating natural-language feedback into bounded edits of MPC weights and bounds, in the spirit of recent LLM-based optimization methods [19], [20] and safety-guarded parameter tuning in autonomous systems [21]. The low-level NMPC then closes the loop with hard feasibility constraints, while the language-driven regulator enables fast adaptation across terrain geometries and task preferences.

In summary, our full contributions are as follows:

• Terrain-conditioned neural kinematics. We develop a map-attached neural kinematics model that fuses terrain and recent motion history to capture hybrid-contact effects without explicit contact-mode enumeration. This yields accurate yet lightweight motion prediction as the foundation for real-time planning.

• Constrained neural MPC. We embed the learned kinematics in NMPC as the state-transition constraint and solve the full problem in 100 ms, enabling receding-horizon control with explicit state and input constraints.

• Language-guided safe retuning. We use an LLM to adapt cost weights, input bounds, and flipper PD gains from high-level feedback, while a safety gate enforces editable-set restrictions, clipping, rate limits, smoothing, and feasibility checks. This improves traversal quality by 67% over PPO and enables terrain-dependent generalization without retraining the predictor or changing the solver.

Related Work

Classical planners for articulated tracked robots with flippers typically rely on geometric reasoning and pre-defined motion primitives. Early systems combined terrain scanning with semi-autonomous flipper control and safety constraints, achieving reliable but conservative behaviors under limited perception and modeling assumptions [8], [14]. Recent geometry-based planners explicitly encode robot–terrain contact relationships to generate real-time flipper motions on rough terrain [22]. Hybrid sampling/optimization methods further improve feasibility on discontinuous, contact-rich surfaces [23], [24], yet the reliance on hand-crafted templates and mode switching restricts expressiveness and can yield sub-optimal trajectories and unnecessary energy expenditure.

An alternative line employs RL. Template-based RL selects among a small set of discrete postures [12], while continuous-action policies learn end-to-end commands [5], [15]–[17]. Despite promising results, RL for flipper robots often requires terrain-specific reward shaping and long training cycles; instability and divergence are common failure modes. The recent FTR-Bench study standardizes tasks and metrics for flipper-track control and highlights the sensitivity of RL performance to reward design and training stability [13]. Moreover, discrete posture switching introduces additional, sometimes pointless, energy consumption.

To balance optimality and data efficiency, learning-augmented MPC integrates learned models into constrained optimization. Real-time neural MPC demonstrates that neural kinematics models can be embedded in real-time control loops for agile platforms [25]. Tooling such as Learning for CasADi facilitates differentiable PyTorch models inside numerical optimization, enabling neural NMPC pipelines with principled constraints and warm-starts [26]–[29]. These advances motivate our formulation, which preserves the structure and safety of NMPC while leveraging data-driven components [30]–[34].

Recent work uses language to condition robot controllers. General VLA models such as RT-2 and OpenVLA learn to map visual observations and language commands to robot actions, improving semantic generalization from large-scale pretraining [1], [2]. However, direct action generation does not by itself impose hard state/input constraints or solve a terrain-dependent optimal control problem. For manipulation, VLMPC parses visual–language instructions and exposes them to an MPC layer that enforces goal and constraint satisfaction [35]. On the learning side, language-to-reward pipelines show that LLMs can synthesize rewards or curricula that steer policy training efficiently and improve transfer [36], [37]. LLMs have also been explored as optimizers, either by iteratively proposing candidate solutions in natural language or by formulating solver-ready optimization models [19], [20]. Closer to our setting, Liang et al. [38] couple language feedback with MPC for data-efficient adaptation, while fielded studies deploy compact, on-board LLMs to perform low-rate, safety-guarded retuning of MPC weights and constraints without disturbing the fast control loop [21]. Our approach follows this pattern: the LLM provides low-frequency, bounded updates to NMPC objectives/constraints, whereas the neural NMPC backbone enforces constraints and closes the high-rate loop.

Methodology

System Overview

The architecture of ASTRIL-MPC (Fig. 2) comprises three tightly coupled layers. First, the front end converts incoming 3D point clouds into a local height map and buffers a short history of recent robot states and control commands. Second, a compiled neural kinematics consumes the height map together with this history to provide short-horizon state increments in a map-attached frame (Sec. III-B). Third, this kinematics is embedded as the model constraint within an NMPC, which solves a finite-horizon optimal control problem with multi-objective costs and hard feasibility constraints, applies the first control input, and then shifts the horizon as new measurements arrive (Sec. III-C). An in-the-loop LLM serves as a parameter regulator: given a compact summary of the recent state–input history and the current parameter vector, it proposes edits to a restricted set of weights, bounds, and flipper PD gains; a guarded interface enforces value ranges, change-rate limits, and basic consistency before any update is applied (Sec. III-D). The full control loop implementing these interactions is detailed in Algorithm 1.

Modeling

State Representation

Explicitly first principles modeling the discontinuous contacts and support-mode switches is too complex to run in real time. We therefore adopt a lightweight neural kinematics model conditioned on a local height sequence sampled from the global map. The state is restricted to the 2D sagittal ($x$–$z$) plane, which is sufficient for capturing the dominant motion of articulated tracked robots (cf. Figure 3). This restriction is also consistent with our platform: the coaxial flipper pair cannot actively control roll, so roll stability must be handled primarily by selecting a globally roll-favorable path. Rather than addressing this global planning, our work serves as a downstream module, focusing on generating the appropriate articulated motions to track predefined trajectories and negotiate obstacles.

At each control cycle $t$, we anchor at the robot’s current longitudinal position $x_t^m$, corresponding to the $x$-coordinate of its center of mass (COM). We take $n_m$ samples ahead and $n_m$ samples behind, spaced by $d_m$, to form an elevation sequence $M_t$ that is fixed over the prediction horizon (H):

Here $M_t \in\mathbb{R}^{2n_m+1}$ is the sampled height sequence (cf. eq:1), where each entry is the ground height returned by $h(\cdot)$ at a longitudinal coordinate. To represent the robot’s position relative to this terrain profile within a prediction horizon, we define a local longitudinal coordinate

Based on these coordinates, we define two types of vectors: (1) a task vector $s_t$ that describes the robot’s absolute state, and (2) a state vector $\tilde{s}_t$ that describes the robot’s state relative to the sampled map, using local longitudinal offsets (anchored at $\tilde{x}_t = 0$), together with their associated control inputs:

for $j = 0,\ldots,H$. To provide the learned kinematics and the NMPC with short-term motion context, we maintain a length-$k$ buffer of recent states and inputs:

where $k$ is the buffer size (number of most recent time steps).

Neural Kinematics Model

We employ a learned discrete-time kinematic model $f_\theta$ that predicts the task-space increment at each stage. The model is queried with: (i) the local terrain sequence $M_t$ held fixed over the prediction horizon (eq:1); (ii) a short history buffer $B_t$ capturing recent motion context; and (iii) the stage control input $u_{t+j}$. The per-stage prediction and update are

where each $\hat{s}_{t+j}$ has the same physical components as $\tilde{s}_{t+j}$ $(\tilde{x}, z, \theta_p, \theta^f, \theta^r)$.

To guide the network toward stable and control-aware predictions, we design a composite loss that balances accuracy, smoothness, and robustness. The details are summarized in Table I. We use a diagonal positive weight matrix $\mathbf{W}_{\mathrm{fit}}$. The positive part is defined as

Term	Definition
$\mathcal{L}_{\mathrm{fit}}$	$\sum_i\left|\Delta s_{i+1}-\Delta\hat{s}_{i+1}\right|_{W_{\mathrm{fit}}}^{2}$
$\mathcal{L}_{\mathrm{over}}$	$\lambda_{\mathrm{over}}\sum_i\left(\Delta\hat{x}_{i+1}-\Delta x_{i+1}\right)^{2}$
$\mathcal{L}_{\mathrm{sm}}$	$\lambda_{\mathrm{sm}}\sum_i\left|\Delta\hat{x}_{i+1}-\Delta\bar{x}_{i}\right|_{2}^{2}$
$\mathcal{L}_{\mathrm{mag}}$	$\lambda_{\mathrm{mag}}\sum_i\left|\operatorname{clip}(\Delta\hat{s}_{i+1};\delta)\right|_{2}^{2}$
$\mathcal{L}_{\mathrm{reg}}$	$\lambda_{\mathrm{reg}}\left|\theta\right|_{2}^{2}$

Elementwise clipping is

The short-horizon forward-motion trend is

where $\Delta x_i$ denotes the $x$-component of $\Delta s_i$. Fit ($\mathcal{L}_{\mathrm{fit}}$) is a weighted MSE on the predicted increments, letting critical channels receive larger weights. Overshoot ($\mathcal{L}_{\mathrm{over}}$) penalizes only positive forward-direction bias $(\Delta\hat{x}-\Delta x)_+^2$, discouraging overly aggressive advancement. Smoothness ($\mathcal{L}_{\mathrm{sm}}$) aligns the predicted forward increment $\Delta\hat{x}$ with the local trend $\overline{\Delta x}$ to reduce jitter across steps. Magnitude ($\mathcal{L}_{\mathrm{mag}}$) penalizes clipped predicted increments to keep outputs within plausible ranges and improve robustness to outliers. Regularization ($\mathcal{L}_{\mathrm{reg}}$) applies $\ell_2$ weight decay on $\theta$ to curb overfitting.

Model Predicted Control

At each control step $t$, the NMPC optimizes over the task-space states $s_{t+j}$ and control inputs $u_{t+j}$ defined in Sec. III-B. It uses the learned kinematics ((8)–(9)) together with the fixed height sequence $M_t$ (eq:1) to solve a finite-horizon problem with multi-objective costs and strict feasibility constraints. Input penalties are centered around a PD feedforward term $u_{\mathrm{ff},t+j}$.

for $j=0,\ldots,H-1$.

Cost and Reference

We use diagonal state weights $\mathbf{W}$ (terminal $\mathbf{W}_{\mathrm{e}}$) on the five task variables and diagonal input penalties $\mathbf{R}$. The input term is centered at a feedforward sequence $\mathbf{u}_{\mathrm{ff},t+j}=[u^{\mathrm{f}}_{\mathrm{ff},t+j},u^{\mathrm{r}}_{\mathrm{ff},t+j}]^{\mathsf{T}}$ aligned to the NMPC grid. For the flippers, $\mathbf{u}_{\mathrm{ff}}$ is generated by per-joint PD laws:

Here $\left(\theta^{\mathrm{f}}_{t},\omega^{\mathrm{f}}_{t}\right)$ and $\left(\theta^{\mathrm{r}}_{t},\omega^{\mathrm{r}}_{t}\right)$ are the measured joint positions and velocities at time $t$, while $\left(\theta^{\mathrm{f,ref}}_{t+j},\omega^{\mathrm{f,ref}}_{t+j}\right)$ and $\left(\theta^{\mathrm{r,ref}}_{t+j},\omega^{\mathrm{r,ref}}_{t+j}\right)$ are the stage-$t+j$ references. To suppress first-stage jerk on the chassis, the forward-velocity reference is ramped within the horizon:

with $v_{0}$ the measured speed at time $t$. Before entering the cost, each $u_{\mathrm{ff},t+j}$ is clipped to the actuator box in (17) and subjected to the same rate limit $\Delta u_{\max}$ (with smoothing factor $\alpha_{\mathrm{sm}}$) as the applied control, which keeps the penalty center feasible and reduces early-stage optimizer effort.

Constraints

We enforce feasibility by clamping the initial state to the current measurement $\hat{s}_{t}$ and constraining all control inputs within a bounded feasible set. At each stage, $u_{t+j}=[v_{t+j},\omega^{\mathrm{f}}_{t+j},\omega^{\mathrm{r}}_{t+j}]^{\mathsf{T}}$ must lie in $\mathcal{U}=\{u\mid u^{\mathrm{lb}}\leq u\leq u^{\mathrm{ub}}\}$, which includes explicit forward-speed bounds $v_{\min}\leq v_{t+j}\leq v_{\max}$. These bounds are part of the editable parameter set (Table II) and can be adjusted online under safety gating.

Language-driven Adaptation via LLM

Although NMPC provides a principled way to enforce constraints and balance multiple objectives, its performance on ATRs is sensitive to cost weights and constraint bounds. This sensitivity becomes more severe under out-of-distribution terrain, where a fixed objective may no longer express the desired trade-off among speed, stability, and impact. Instead of using an LLM as an open-loop action policy, we use it as a structured optimization assistant: following the idea that LLMs can propose and refine optimization variables or solver-ready formulations [19], [20], the model only suggests parameter edits for an existing constrained controller. This yields a rapid and interpretable adaptation layer without altering the NMPC formulation or bypassing its feasibility constraints (Figure 4).

When a new operator instruction arrives, the controller assembles a compact prompt $Q_{t}$ from: (i) the current measurement $(\hat{s}_{t},u_{t})$; (ii) the length-$k$ history buffer $\hat{B}_{t}$; (iii) the active parameter vector $\lambda_{t-1}$ that instantiates the NMPC weights $(\mathbf{W},\mathbf{W}_{\mathrm{e}},\mathbf{R})$ and bounds $(v_{\min},v_{\max})$; and (iv) a concise parameter dictionary and optimization template (objective and constraints in (13)-(18)), followed by the operator instruction $\operatorname{instr}_{t}$. The prompt therefore asks the LLM to solve a small, semantic parameter-selection problem rather than the continuous NMPC itself. The LLM is treated as a conditional generator of parameter updates; its output is restricted to a predefined subset and passed through a safety gate that enforces bounds, per-update rate limits, and smoothing before any change takes effect.

Algorithm 1 LLM–NMPC pipeline

1. Offline: $M \leftarrow\operatorname{ExtractHeightMap}(\mathcal{P})$; $\{s_{t+j}^{\mathrm{ref}}\}_{j=0}^{H} \leftarrow\operatorname{GenerateReference}(M)$

2. Init: buffer $B_t$, model $f_\theta$, params $\boldsymbol{\lambda}_0$, horizon $H$

3. while robot active do

4. Update $B_t$ and local map slice $M_t$

5. if new instruction then

6. $Q_t \leftarrow\operatorname{BuildPrompt}(\operatorname{instr}_t, M_t, B_t, \boldsymbol{\lambda}_{t-1})$

7. apply guarded update (15a–15c)

8. end if

9. Build PD feedforward and ramp $v^{\mathrm{ref}}$

10. for $j=0$ to $H-1$ do

11. $\hat{s}_{t+j+1} \leftarrow\hat{s}_{t+j} + f_\theta(M_t, \hat{B}_t, u_{t+j})$

12. $\hat{B}_t \leftarrow\{\hat{s}_{t-k+1:t},u_{t-k+1:t}\}$

13. end for

14. Solve NMPC (11a–11f); apply smoothing; send $u_t$

15. end while

Prompt construction and parameter extraction: Using the above quantities, prompt construction and parameter extraction are written as

where $\mathcal{S}$ is the editable set in Table II (state-tracking weights, input penalties, flipper PD gains, smoothing factors, and velocity bounds). All other components remain fixed.

Category	Parameters
State tracking weights	$W_{\mathrm{fwd}},W_{z},W_{\theta_{\mathrm{p}}},W_{\theta_{\mathrm{f}},W_{\theta_{\mathrm{r}}}}$
Control penalties	$R_{v},R_{\omega_{\mathrm{f}}},R_{\omega_{\mathrm{r}}}$
Feedforward shaping	$K^{\mathrm{f}}_{\mathrm{P}},K^{\mathrm{f}}_{\mathrm{D}},K^{\mathrm{r}}_{\mathrm{P}},K^{\mathrm{r}}_{\mathrm{D}}$
Smoothing and limits	$\Delta u_{\max},\alpha_{\mathrm{sm}}$
Velocity reference/bounds	$v_{\mathrm{ref}},v_{\min},v_{\max}$

Guarded update: The suggested edits are passed through a safety gate that enforces validity, magnitude limits, and temporal smoothness:

with elementwise bounds $(\boldsymbol{\lambda}^{\mathrm{lb}},\boldsymbol{\lambda}^{\mathrm{ub}})$, a per-update $\ell_\infty$ cap $\Delta u_{\max}$, and smoothing factor $\alpha_{\mathrm{sm}}\in(0,1]$. Feasibility checks (e.g., $v_{\min}<v_{\max}$ and $\mathbf{W},\mathbf{W}_{e},\mathbf{R}\succ0$) are verified before applying (15c). Only $\boldsymbol{\lambda}$ is updated; the perception stack, the learned kinematic model $f_\theta$, and the NMPC solver remain unchanged. This mechanism enables fast, context-sensitive retuning from high-level feedback while preserving the underlying control pipeline.

Runtime loop. Algorithm Figure 1 summarizes the closed-loop procedure. Offline, a local height map and reference profiles are prepared (Step 1–2). Online, each cycle updates the history buffer $B_t$ and the fixed terrain slice $M_t$ (Step 4). If an operator instruction arrives, the controller freezes actuation, builds a prompt $Q_t$ (Step 6), parses the LLM output and applies the guarded update (Step 7). It then constructs the flipper PD feedforward $\mathbf{u}_{\mathrm{ff}}$ (Step 9), rolls the learned kinematics over the horizon (Step 10–12), and solves the NMPC (Step 14).

Experiments and Results

Experimental Setup

Platform and Simulation Setup

We use the tracked rescue platform from [39] (Figure 1), featuring four independently actuated flippers and a symmetric chassis. Simulation scenes follow [22]; the robot traverses terrains represented as height maps with a sampling interval $d_m=0.0625$ m (1a) and the full NMPC loop runs within 100 ms per …

Hardware and Software

All components run on ROS Noetic, Ubuntu 20.04. The NMPC is modeled in CasADi [27]; the learned kinematic model is compiled to solver-compatible code via L4CasADi [25, 26]; and the resulting nonlinear program is solved online with ACADOS [28]. Experiments run on an Intel i5 CPU with an NVIDIA RTX 2080Ti GPU.

Default Parameters

Unless otherwise stated, we use horizon $H=12$ with $\Delta t=0.1$ s; state-tracking weights $W_{\mathrm{fwd}}=3.0$, $W_z=0.5$, $W_{\theta}=15.0$, $W_{\theta_f}=5.0$, $W_{\theta_r}=5.0$; input penalties $R_v=10^{-3}$, $R_{\omega}=2\times10^{-2}$, $R_{\omega_r}=2\times10^{-2}$; flipper PD gains $K_P^f=K_P^r=2.0$, $K_D^f=K_D^r=0.3$; rate/smoothing $\Delta u_{\max}=0.18$, $\alpha_{\mathrm{sm}}=0.8$; and speed targets $v_{\mathrm{ref}}=0.30$ m/s with initial command $v_0=0.12$ m/s. These categories correspond to the tunable entries in Table II.

Language Model

Text-conditioned adaptation is realized by the DashScope Qwen-Plus LLM [40], which generates bounded updates to NMPC weights and soft constraints from sliding-window state summaries and high-level task instructions.

Data Collection and Training Details

We train a next-step kinematic predictor as supervised regression that maps a history-augmented input to the 5-dimensional task state at $t+1$. The input concatenates a 64-bin local height sequence, recent longitudinal positions, recent commands, and short-horizon robot states (Sec. III-B); samples are recorded at 10 Hz and post-processed into the compact representation.

The dataset comprises 5k simulated samples from stair ascent/descent, split 95:5 for training/testing. The predictor is a feedforward MLP with residual connections and a linear 5D output head, trained with Adam (lr $1\times10^{-4}$, batch 2048) using the control-aware loss in Table I; generalization is evaluated separately in Sec. IV-F.

Evaluation and Analysis of Neural Kinematics

To isolate the effect of the loss (symbols as in Table I), we compare three configurations: Type 1 ($\mathcal{L}_{\mathrm{fit}}$ only), Type 2 ($\mathcal{L}_{\mathrm{fit}}+\mathcal{L}_{\mathrm{over}}+\mathcal{L}_{\mathrm{mag}}+\mathcal{L}_{\mathrm{reg}}$), and Type 3 (Type 2 $+\mathcal{L}_{\mathrm{sm}}$). Here $\mathcal{L}_{\mathrm{fit}}$ is the fitting term, $\mathcal{L}_{\mathrm{over}}$ penalizes forward overshoot, $\mathcal{L}_{\mathrm{mag}}$ constrains prediction magnitude, $\mathcal{L}_{\mathrm{reg}}$ is the $\ell_2$ regularizer, and $\mathcal{L}_{\mathrm{sm}}$ encourages temporal smoothness. All models are trained and evaluated under identical data and rollout settings.

Evaluation Protocol and RMSE Aggregation

Let the prediction horizon be $H=10$. For each evaluation trajectory of length $T$, we split it into $S=\lfloor T/H\rfloor$ non-overlapping windows of length $H$. At the start of each window we freeze the height sequence and history buffer, run the model open-loop for $j=1,\ldots,H$, and measure step-ahead errors on the task-vector components $c \in\{x,z,\theta_p,\theta_f,\theta_r\}$. For each step $j$, we compute the RMSE across the $S$ windows between the $j$-step predictions and ground truth for each component. We compare the per-step RMSE across the $x$ direction, the $z$ direction, the pitch angle, and the front/rear flipper angles.

Results and Analysis

As shown in Figure 5 and Figure 6, Type 1 exhibits horizon-wise drift, Type 2 reduces this drift but still shows noticeable errors, and the full loss (Type 3) yields the most accurate and stable rollouts across all state components. Meanwhile, the trajectory visualization shows the same trend: Type 1/2 diverge from the ground truth as the horizon unfolds, whereas Type 3 stays closely aligned throughout the rollout, supporting the effectiveness of our final loss design. The RMSE curves and the visualization are consistent across all variables, indicating that the errors are not dominated by a single state.

Ablation Study of LLM Adaptation

We provide a qualitative ablation to highlight how language instructions affect the closed-loop behavior; quantitative effects are discussed in the comparative experiment subsection. As illustrated in Figure 7, enabling instructions (Figure 7) allows the LLM regulator to perform bounded retuning of selected NMPC weights/bounds, resulting in different flipper configurations compared with a fixed-objective controller. Moreover, instruction correctness matters (Figure 7): correct instructions guide the controller toward the intended trade-off between stability and impact while preserving feasibility under hard constraints, whereas incorrect or mis-specified instructions can bias the regulator and induce unnecessary configuration changes or increased shocks. This ablation suggests that retuning influences the stability–impact trade-off in a consistent manner across instruction settings.

Comparative Experiment

We compare against a reinforcement learning baseline implemented with Proximal Policy Optimization (PPO) [13]. Figure 9 highlights the macro-level effect: the LLM-retuned controller maintains smoother body attitude across terrain changes, with fewer abrupt pitch/roll excursions at edges and transitions. This is visible as more cautious edge approaches on the single-block case and steadier step-to-step motion on stair ascent/descent.

Evaluation Tasks

We evaluate three scenarios: single-block crossing, stair ascent, and stair descent. Each scenario is executed three times per controller, with success defined as reaching the goal within 60 s without tipping.

Performance Metrics

To evaluate traversal quality, we record the following six raw indicators over each run: $\phi$, $\theta_p$ are the roll and pitch angles (in rad), $\dot{\phi}$, $\dot{\theta}_p$ their angular velocities (in rad/s), and $a^x$, $a^z$ are the linear accelerations of the robot center of mass along the forward and vertical axes (in $\mathrm{m/s^2}$). For clarity, we group them into three paired terms that summarize stability, smoothness, and impact:

Each pair of indicators is normalized to $[0,1]$ and combined into three dimensionless penalty terms (lower is better). These penalty terms are broadly applicable to most prior studies on ATRs traversal, as they capture stability, smoothness, and impact in a task-agnostic manner. All experiments are repeated three times per scenario, and the reported values are the means across repetitions.

Results

The comparative results across terrains are shown in Table III. To consolidate the evaluation, we average the three normalized scores defined in eq:16 into a single value per terrain and then compute the mean across all terrains. Overall, NNMPC+LLM(Ours) achieves a 42% better mean score than NNMPC(Ours) and a 67% improvement over PPO. For the key stair scenarios, the mean score improves by 33% on ascent and 71% on descent relative to NNMPC(Ours). On the single-block case, NNMPC(Ours) is slightly higher on the mean score, yet the adaptive variant markedly suppresses impact (Shock $1.4 \times10^{-4}$ vs. $3.0 \times10^{-3}$). The gains are driven primarily by reductions in stability and shock penalties, indicating that online weight adaptation improves the stability/impact trade-off, especially on the more dynamic stair scenarios.

Terrain	Method	Stability	Swing	Shock
Single-Block	NNMPC + LLM (ours)	0.39	0.63	$1.40 \times10^{-4}$
Single-Block	NNMPC only (ours)	0.50	0.33	$\underline{3.04 \times10^{-3}}$
Single-Block	PPO	0.50	0.50	1.00
Stair Ascent	NNMPC + LLM (ours)	0.37	0.08	$\underline{2.96 \times10^{-4}}$
Stair Ascent	NNMPC only (ours)	0.66	0.00	0.00
Stair Ascent	PPO	0.50	1.00	1.00
Stair Descent	NNMPC + LLM (ours)	0.49	0.12	0.00
Stair Descent	NNMPC only (ours)	0.56	0.98	$\underline{0.57}$
Stair Descent	PPO	0.50	0.50	0.76

Generative Experiment

We further evaluate generalization by testing the proposed framework in a single environment containing terrains with multiple obstacle heights (Figure 8). Without any task-specific re-training, the controller maintains feasible configurations across height variations, indicating transfer of the learned kinematics and the language-driven retuning mechanism to unseen geometric conditions under a fixed deployment pipeline. Across increasing obstacle heights, the system keeps the chassis within a bounded pitch range by modulating front-flipper probing and rear-flipper bracing; this yields consistent contact sequencing and avoids hard impacts at step edges. The same tuned parameter set is reused across all height regimes, and the LLM’s bounded updates remain stable, suggesting that the conditioning on recent history and terrain profile provides sufficient context for safe retuning under distribution shift.

Conclusion

We proposed ASTRIL-MPC, a framework that brings the strengths of NMPC to ATR traversal while addressing practical modeling and tuning challenges. By combining a neural kinematics model with a language-driven LLM regulator, our method reduces the burden of scenario-specific parameter tuning while preserving feasibility and real-time performance. Furthermore, compiling the neural kinematics into the solver accelerates optimization, enabling real-time control on ATRs operating in structured, contact-rich environments. Our experiments indicate that the learned kinematics and composite loss improve multi-step prediction quality, the loss design stabilizes rollouts by curbing overshoot and encouraging smoothness, and language-driven retuning yields more stable traversal across varied terrains.

In future work, we plan to extend ASTRIL-MPC toward handling deformable or variable-geometry objects and incorporating energy-aware objectives to further improve efficiency in long-duration missions. We also aim to validate the framework on additional real-world platforms and expand the range of terrains and contact conditions.

References